CRESST REPORT 789 


Denise Huang 
David Silver 
Mandy Cheung 
Nikki Duong 
Alice Gualpa 
Cheri Hodson 


Deborah La Torre 


Matrundola 

Nora Obregon 
Jordan Rickles 
Gwendelyn Rivera 
Yulin Sun 

Larry Thomas 


Vanessa Vazquez 


INDEPENDENT STATEWIDE 
EVALUATION OF AFTER SCHOOL 
PROGRAMS 


APRIL 2011 


The National Center for Research on Evaluation, Standards, and Student Testing 


Graduate School of Education & Information Sciences 
UCLA | University of California, Los Angeles 


Independent Statewide Evaluation of After School Programs 
ASES and 21st CCLC 
Year 2 Annual Report 


CRESST Report 789 


Denise Huang, David Silver, Mandy Cheung, Nikki Duong, 
Alice Gualpa, Cheri Hodson, Deborah La Torre Matrundola, 
Nora Obregon, Jordan Rickles, Gwendelyn Rivera, Yulin Sun, 
Larry Thomas, and Vanessa Vazquez 
CRESST/University of California, Los Angeles 


April, 2011 


National Center for Research on Evaluation, 
Standards, and Student Testing (CRESST) 
Center for the Study of Evaluation (CSE) 

Graduate School of Education & Information Studies 
University of California, Los Angeles 
300 Charles E. Young Drive North 
GSE&IS Bldg., Box 951522 
Los Angeles, CA 90095-1522 
(310) 206-1532 


Copyright © 2011 The Regents of the University of California. 


The work reported herein was supported by CDE grant number CNO77738 with funding to National Center for 
Research on Evaluation, Standards, and Student Testing (CRESST). 


The findings and opinions expressed in this report are those of the authors and do not necessarily reflect the 
positions or policies of CDE. 


To cite from this report, please use the following as your APA reference: 

Huang, D., Silver, D., Cheung, M., Duong, N., Gualpa, A., Hodson, C., ...Vazquez, V. (2011). Independent 
Statewide Evaluation of After School Programs ASES and 21st CCLC Year 2 Annual Report. (CRESST Report 
789). Los Angeles, CA: University of California, National Center for Research on Evaluation, Standards, and 
Student Testing (CRESST). 


TABLE OF CONTENTS 


PTD SUT ACU pes ca te cnuenzlv cas appease paca clv scab bee wea Sai ca aa eaed VS ao ec ce Seay ave cea 1 
THPOCICE OM; 2.545 iy cdesadycn sedis hang easniyaas apes juan s bocpieety dal daa buadeanapne yadenbasaaallagendiearpandeundiuc lela dumantenceene il 
Chapter I: Theoretical Model of the Study ..............:scssccssccssscessecensscssccssccessceenssesscessncesscesnasenes 3 
Evaltiation QueStiOns ..........cccccccccccesssssssssssccscesecesesssseesscsscssesessuseessssseesscssacssesessesuesscesseescess 4 
Chapter II: Establishing a Sampling Frame ..............cccecccesssceesseeeeeceeeceeeeeeaeecsaeceaeceeeeeeaeeeaeensaeees 5 
PVA SOURCES Cocoa chalets Etec taat hea ttes be eine ote ectidentz cis vescand tenn SG caactadocadecat ee rade caddy gous fs) 
SIN TS SCT Co estan teal atc Bae catn hic i Sete uc eae as itd in deel Uaioe ah 5 
SATIS CS1 Ost sacat ac net unaterai encse-ro aeansgiats sn nezueasionevennasaetdes ue euaennse tug enueee enssh deus oa ontuneeds 5 
SATU C dst te, sitet ss uiecca rs est cucssiocveucessevedass ute. eoey du ctesr ous acs acne dosy ebacwues ebeceaseere ucts: oretecoubecausouveee 6 
SATIN Le efecs tates osetia riage cacti aal eet ocala tint ten eel eca tt nate tie cence leat tbe ce ited 7 
Bye I LEN ss ea cue sah se catocueseeslya teeta dusocsecue unas saree ena auedssnvasecese sesteateie ween aaeieeenaieas anes 7 
SATIN IV. reeset Pore aS eaneseacuptd vveed a ndecadiiadc tenets scmegnedenseedeas anaes dimacoebneegtinertesdes woe Mad vedsceee: 8 
Chapter ITT: Data Collection in Year 2.............:scssscssscssscesssessccssccesscesnssescessccesscesnsenscessecesnees 10 
STEVI GP atinet Feet sts care ceeue tee des anand saeco sotatel cee eee detunticod coet oa Reaenac stand aeeadatagh eed pa coonecetale teshende 10 
pe Hea] aL ener eee mere eer oe eee Ory PEE Ha PeMnne DR ove een per an Aten Top a Ey Seroey Ter a Meee enor 10 
SATS TN se cnstevs testes sects ech sa savegniend devsekd ont svecde chet aacavevsacedlenseacottyuacdathevascentenieedteyseideus esaasiee 11 
Data-CollectiOneProcesg x ves sc9ts2idevedndeces lasesorevegcansiavense open eed baa rensnven cab daviese niece steeples 12 
SATIN NW paths OeceteesS ine Move tae ee oapree e nyetra eL h Oe ea eae Ma Ae ME Mie uasee tases 13 
Bata Collect ne Process a ass sec: ac cance eas Saseticee he cose latat ca agctoode ate neve etaentd nee cenuareaededecameramos 13 
REGtIIIMENE OF Pantie PAnts2 2.0 tes of ots cece ween Rte cat sda nutes, eet cuaec at cress 14 
Chapter IV: Preliminary Estimates of After School Participation Effects ........ eee eeeeeeeeeees 16 
Levels of After School Participation and Frequent Participants..............c:cccsseceseeeeeeeeeeeees 18 
Academic Achievement Outcomes (Sample [)...........ccccsceesseceseceseceececseeceeceeeeeeseeeaeceeeeues 19 
Performance On the CST. sie cciesiseseetuadeaigees cdaveni nics bindeawigendedesdacseutiinndthahatinanneateiatos 19 
Pertormance on theCRL DY s.t05 ccccecssceics ets haste aeeccare ated eee ne ciesv aeeess e 21 
Behavior Cute Omes sss cssnecg tented cadet cat eceeudnaegcdes one en aaihnuc ise cates det edd eeruak uatinc sds eecte ea 23 
Phiysieal Bitmesss( Samp bes Loess datase a ccadecss A aitandse crude ecsddcahvasdaciercc nditec saat aa deumecasaeunatce 23 
School Day Attendance (Sample I) .........ccceesceceseceseceeseeeececeaeceseceeeeeseeeeeseeesaeceeeeeeeeeaeees 26 
School Suspensions (Sample I) ..............ccssscssscesessessccssccescesessenscesseceseccesscesensencessecesneees 28 
SUPE Vi seavteodecaaaelcs caalcawosasaatden Gace duces audes Adin sana tyans cna vag dacori death mshi aerate meade 30 
Chapter V: Descriptive Results of Year 2 Grantee Profiling Reports (Study Sample 
VID) scg tes staasscusaeactiresudieacuncuseatstaveestanedscystadsetavieansaw isis Sauedactastas adheiaeeioaiteeeretneaatis Be 
BUmidliie SOUECES ss sco) oe es encased cctvernadadreeseas wera dccecseey atied ie eeaieecee se ated decee RG antag 34 
Fee Scale and Enrollment Strategies at Sites ...............ecsssssssscsssscseccssecesscsensscscessccessesensens 36 
Student Recruitment and Retention ...............ssccsscsssrssssccssccesscesessssccssccesscessssenscessseessncesneees B7 
R GCC MO@NE OF Stitt OI Sse vests secreted yaucce rgesvedesseceouct sess obecs aunceuea pe cucsveueauce gen cles pycsanvends 37 
RetentiOn OF STUGents ci ysecocssincedisvesccas sac becienetens sssece dha vsbsvasyaectacioutacoad i teeaelenee nto thatoatviess 38 
Gals Arid: OUtCOnies:, 227, A.veasccesassepcnsaca sass iaactsshaausceers seed Waduesenaeausastereseueds atta vaantimaeaniataress 38 
Resources for Setting Goals and Determining OUtCOMES............ceceseeseeeeeseeeseceseeneeenee 38 
Goal Setting and Outcomes at the Grantee Level ...........cccceesseesteceeeceeeeeesceceaeeeseeeeseeeenes 39 
Goal Setting and Outcomes at the Site Level 0.0... ee eeecececceeseeeseeceeeeceeceeeeeeaeecseeceteeeenes 40 
Prop ra mints ane PVCU Viti OS sts rcns ale ac siccchen toe cca talres ical tourettes sce mania llch Soa atety aston 40 
Prog ramming amp Masse oisc.asveadedcsaecas tee ca scesacancs eaacvetanseva sede tecusct a seecuanepe ecto eda presntecat aaaey 41 
PNCACEMIG A CUN IES 5 «este Se canedtedeed dav cn dual doe veapeabieesesiaedauteswcenl daxsetveh deudessiwcenl Gasveouamaneen 41 


iii 


Non-Academic Enrichment Activities ............cccccccssssssssssssssssssssssssssssssssssssssssssssssssssssssenes 42 


SERENA sce chords carn wet exscuiee tnces sanetes, eevcadey pc elice oukGaat ce deebets ones edtaasvaustaeeeGaitcecredeleesaceetyabees 42 
Statt Quali fiGatiOns 5 schieis hiss eee assesses hada Shas 43 
Site Wand sement vice, edie sead aoa iene aa alae ene amarmneeeus 43 
Types of Staff Employed and their Retention. ................:ccscccsscsssrssssccsscessceseesssscessncesnees 44 
Techniques used to Recruit Staff by Urbanicity and Grade Span ........ eee eeeeeeeeeseeeeeeeee 44 
Techniques used to Retain Staff by Urbanicity and Grade Span......... ees eeeeeeeseceeeeseeeeee 45 

Proresstotial Dev elOmimie nit soy iced wok cede dca Nearest cg ace tcl nd ae och bad cata deak acetate condita area 45 
Staff Offered Professional Development .............:.ccsccssseceseceeeceeeeeeaceceaeceeceeseeeaeecsaeeneeees 45 
Organizations Providing Professional DevelopMent...............ssesseeseceeeceeeenceeeeeeeeeseeeaees 46 
Types of Professional Development .............:.ccccceessessteceeeceeeceeseeeseeceseeceaeceeeeeeaeecsueeeeeeeees 47 
Professional Development Topics ............c.ssccsscssssssssccssscessccsecessssessccssccesscessssenscessnceencens 47 

Comiunitinity Partnerships iercc«, icths tacencgt cp saeeeaecceesoutend sss eceesweresanes vei Santer betwee twee 48 
Community Involvement by Urbanicity and Grade Spann... eee eecesecseenceeeeeeeeeeeeaees 48 
Individuals and Organizations that Partner with the Sites... eee eeeeeecesecneeereeeeeeeeees 49 
Different Roles at the After School Sites...............scccsscscesrcesecescrssseerseocesncetecoacescenorenceonee 49 

SULEIIIIIALY <sccisires panic dandearepiank dhs Soanuwd esos eben haavenunscasadenbandeveismleratiaddoiniseadedesdanwbipaderannaaniagees 50 

Chapter VI: Descriptive Results of Program Sites—Sample IV 0.0... eeseeeeeseceseceeeeneeeneeeeeees 53 

Sample TV Respondent Characteristics...............scssccssscssscscssscssccssccesscesensesscesencesscesnesencesaes 53 
Siselenit © Haracheri st CS seca sna cascecverecteaierascassatocscedshatiaduaasesowsysacescyieseneea maeancromeaens 53 
Parent: GHaraCteris ties x5cacecsinadcs uniter cevaesbeneais vay sassy cesviabecciveerg ease b's ico lsdenseon ers caieunywertenras 55 
Statt Qualifications sii2etesiesie see ehaieesee ate Madi RS he oe ae Se 55 

Pro grat: EW VITOMMENU ¢ saiaeectazvecaanl veeniceesden se dicalessactutes datvees ncacedeada tuaedcasasucenesmeepaaigvee acne 63 
PRET AUIO tis Hil [9S enresace ates ec rete tas te ceatreeaessde aah eeuaden sts eat eeceet de hekdcon aed be raat eeecrei side, eee 64 
PRES OUNCES 6c chic ested ect otatiutastectlne ts kevsseteche chases eeusteceGi eae ede teeta eeieae les eaaoaea vs 66 

TS TPRICEL Oia PGA OS  cccsaclese cuted cncaziiey ca eaceuvncias ver seeP apse andes $d ov ies cease caves decease ep Mossy escapees: 67 
SLATE PEC AC otc eiectt 08 des Seal tlt NE Soa ai tc Male a eae hea Es Sach als hoa dy 67 
SEP PORE ORE CAC Y pesca cscaceeniecetden ese eadoe atndats veuteansaseics 250 ter ne ca Pageseseudtons cuvauecataecdevanedt 68 
Professional Development .............:ccsscssccssscessccssscsscsssccesscesnssescessccesscesesesscessncesscesnasons 69 

Fostering Positive Youth Development ...............:ccsccssccsscessceeeeecescesseeseccesseeesseceseeseeeeeaeees fal 
Safety, Caring Relationships, and Belongingness ...............ccscceescsesceeseeceeceeeeeeeeeeaeeeaeeees 73 
Appropriate Structures and Positive Social NOrmMs............ccsesessceseeesecesecneeeneessecsaeeneeenee 73 
Opportunities for Skill Building, Support for Efficacy and Mattering «0.0... eee 73 
IAT Gi OC LAU ONS fice cPets canes tacts reece avec pamic utes: Medico c nates Scie bund eae aluceuareusbbessuametec Aves 75 
PINCUS PASP TALL OMS rads tyveciscacsntvnsdeneeetacnstieus daseteaseoytvacd inves sesenedeanpieaonnuetasecuhatiag towiuteasyaaias 76 
Other Aspects of Positive Youth Development ................cesceseesceereeeeeeseceseceaecneeeneeseeenaes 76 
Integration of Family, School, and Community Efforts 00.0.0... ee eeeeeeeeceeceeeeeeeeeeeeees 76 
Academic Self-EthCacy cit ssccveisasessaledonsnussvnntesusneastbedgoavededdocaosboaeeeedooesenes Seneeubavecleensdoupets Af 
PLCACOINIC PCIE sont donc cerndes are og lang sabi cesaucuo.nexeaudstuasatad sau pudh tasaucenme ceaudeteaerems aiete ase: 78 
PACACOTIC, SILLS se asecace dats tevecatscsunan leona fonaconasseatea Ma utans sat uct buatetatia ua tu umcet pace tawny 78 
GOgMItIVE: COMPOteNCe jisecisyissdsivveced sens sbdaus Goan ianessunssanscddeuddunceuslueedsusavenaancies oblast sueascbvvasdass 80 
Soci0-E motional. GomipetenCe iio:32.cged.sdeceslascacoseeeeecaguceveswsesecwsees tae revs eaves canedaetdedantecteueees: 81 
BG lOM SS ANG ATEN ANC OS 0 a alee sue cs inte se tegate eee A oetencest eh castes elect cd vramioraiied Glee’ 83 
Berio Sil S anid HO WEG Be act oan A Sars ede ce cutedesatand seg oededata ue heans ueletauertncsSoedesetaavessamitens 84 
PP UTUS PASIO AON Gaccht a aerate Peoria ne Weta etaG ve ua ee Ne nee ast ealle ttre att Ae laed raat tate ae ce 84 

Evaluation System and Use of Data for Continuous Improvement..............:ceseeeeeeeeeeeeees 85 


iv 


General Satisfaction OUtCOMES ..............cccecesesesesesesesecesecesecesesseesssesesscessescscssssssssssssesesecesesece 87 


Slee SACIStACT OM x cco, cs ecusseeeeaceseaped ese ores seapeonctea oertactuqarekesevuneaataaguencgveuneaesuasnedeiewen, Gseas 88 
Perceived Recruitment Obstacles .............ccsccscessscsccssssoceonccnscsacesscensscncsonceeevactscenetentevacees 89 
Perceived Obstacles for Participation ............ccccescesecesceeseecesceceseceeeeeeeeeseeesaeceeeeeeeeeaeees 89 
SUITED Yc ast Baila eules cag Staiv ae weed Petes Bidet d ede peshide dd ude owearens SU dena apeudensaddac nincudetes 90 
Chapter VI: Summary of Year 2 Findings............ccccescsseceeceeseeceseceeeceeeeeeseecaeceeeeeeeeeeeeeeneees 94 
Looking Ahead: Roadmap for Year 3.............sccssscssssssssscssccssscesecsensscsecessccesscsenssesscessecesseees 95 
FRET ORCI CS saryes pacietsostatatanan tir ceten erst sagas Wicaueitai ees na Seuaans dantudlih auecatatia te cakematyttaaadean nectar: 97 
Appendix A: Summary of Data Sources and Profiles of Student Characteristics 
CGA tC ITE eas etea Aedes cred needa IOS ces ean er dh cenaesSeedanidl eco nimegesbnderebaveaced eaves 101 
Appendix B: Student and School Characteristics (Chapter [V)............ssssseeseseeeseceseceeeeneeees 119 
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Abstract 


After school programs offer an important avenue for supplementing educational 
opportunities. In California, the After School Education and Safety (ASES) program 
creates incentives for locally driven after school programs to partner with schools and 
communities in providing academic support and safe, constructive alternatives for 
elementary and middle school students. This paper presents findings from Year 2 of a 
four-year longitudinal study describing the statewide landscape of the California ASES 
programs. It examines the effectiveness and efficiencies of these programs in recruiting 
and retaining students at risk and in increasing their academic successes as indicated by 
California Standardized Test scores, youth development outcomes, and the English 
learers’ California English Language Development Test scores. 


INTRODUCTION 


The purpose of this report is to present the preliminary Year 2 findings of a four-year 
longitudinal study on the California statewide after school programs funded by the California 
Department of Education (CDE). Since the provisions of Proposition 49' became effective 
with the passage of the 2006-07 State Budget, Senate Bill 638 became the implementation 
legislation signed by Governor Schwarzenegger on September 21, 2006. As a result, total 
funding for the ASES (After School Education and Safety) program increased from around 
$120 million to $550 million annually. One of the stipulations of this funding is that the 
California Department of Education shall contract for an independent statewide evaluation on 
the effectiveness of programs receiving funding. The National Center for Research on 
Evaluation, Standards, and Student Testing (CRESST) has taken on the responsibility of this 
task and is currently responsible for conducting two statewide evaluations of after school 


programs: one for programs serving elementary and middle school students (ASES and 21st 


‘In 2002, California voters passed a ballot initiative called Proposition 49, which was sponsored by now- 
Governor Arnold Schwarzenegger to increase the state’s investment in after school programming. As it is 
written, Prop 49 provides funding to allow every public elementary and middle school in California to access 
state funds for after school programs. 


CCLC [21° Century Community Learning Centers] programs) and the second for programs 
serving high school students (ASSETs program). This report summarizes activities 
conducted during Year 2 of the four-year study on the statewide evaluation of after school 


programs serving elementary and middle school students. 


In general, this report follows the structure of the Year 1 Annual Report. While the 
Year 1 report focused on the analysis that generated the sampling frame and the procedures 
on the development of study instruments, this report also presents the preliminary analysis 
and findings collected during Year 2 of the study. In this report, Chapter I presents the study 
theory and the evaluation questions that guide the study. Chapter II provides descriptions of 
the data sources, sampling structure, and sampling design for Study Sample IV. Chapter III 
introduces the data collection methodology in Year 2. ChapterIV presents preliminary 
findings on the academic and behavioral outcomes for Study Samples I and II. Chapter V 
describes the Year 2 findings of the effects of attending after school programs for Study 
Sample II. Chapter VI contains the thorough analysis of Sample IV findings from the 
sampled program sites in Year 2. Finally, Chapter VII summarizes the findings from Year 2. 


CHAPTER IT: 
THEORETICAL MODEL OF THE STUDY 


It is essential that an evaluation of after school programming be rooted in the research 
on effective, high-quality program provisions. Based on the extensive literature review 
conducted last year, the following theoretical model was developed. 


Student Engagement } 


aa om. 


rientation 
External Onlenieie 


Connection 


Satisfaction 


Academic 


Program 
Climate 
Relationships Program 
Environment 


School 
Attendance 


Instructional 
Features 


Alignment Staff Efficacy Professional 
Development 


Figure 1. Theoretical model. 


As specified in the model, efficient organization, environment, and _ instructional 
features are crucial for maintaining high-quality after school programs. Having a strong team 
of site staff who are qualified, experienced, committed, and open to professional 
development opportunities is also critical for a successful organization and an overall high- 
quality program. Beyond site staff, involvement of children’s families and communities can 
enhance the after school program experience, foster program growth, and increase program 
sustainability. In order to gauge program success, consistent and systematic methods of 
evaluation are important; they ensure that students, families, and communities involved in the 
program are being effectively served, and they create opportunities for the program to 


continuously self-improve. 
Evaluation Questions 


The following evaluation questions (per Education Code Sections 8421.5, 8428, 
8482.4, 8483.55©, and 8484), together with the theoretical model, guide CRESST in meeting 


the purposes of the comprehensive independent after school program evaluations. 


1. What is the impact of after school programs on the academic performance of 
participating students? Does participation in after school programs appear to 
contribute to improved academic achievement? 


2. Does participation in after school programs affect other behaviors such as school 
day attendance, homework completion, positive behavior, skill development, and 
healthy youth development? 


3. What are the similarities and differences in program structure and implementation? 
How and why has implementation varied across programs and schools, and what 
impact have these variations had on program participation, student achievement, 
and behavior change? 


4. What is the level of student, parent, staff, and administration satisfaction concerning 
the implementation and impact of after school programs? 


What is the nature and impact of organizations involved in local partnerships? 


6. What unintended consequences have resulted from the implementation of the after 
school programs? 


CHAPTER II: 
ESTABLISHING A SAMPLING FRAME 


This chapter provides descriptions of data sources, sampling structure, and sampling 


design. 
Data Sources 


Similar to Year 1 of the study, data sources for this study include: 


1. Student-level academic assessment results and demographic data provided annually 
by the CDE, including data from the following: the Standardized Testing and 
Reporting Program (STAR), the California English Language Development Test 
(CELDT), and the California Physical Fitness Test; 


2. The CRESST-designed “After School Profile Questionnaire,” which is sent to all 
participating after school programs annually to gather in-depth information about 
their program orientation, environment, and instructional features; 


3. Annual collection of student-level behavioral data on sample of 100 districts for the 
elementary and middle school after school programs identified by CRESST; and 


4. In-depth data collection on a subsample of 40 ASES and/or 21st CCLC after school 
sites during Years 2 and 3. The CRESST-designed instruments include surveys and 
protocols for focus groups and interviews. The instruments explore program 
orientation, program environment, instructional features, and level of student, 
parent, staff, and administration satisfaction concerning implementation and impact 
of after school programs. 


Sampling Structure 


This study is designed to utilize administrative data collected by the CDE and school 
districts (secondary data sources), as well as new data collected by the evaluation team 
(primary data sources). The secondary data sources are intended to provide student-level 
information pertaining to after school program participation, demographics, grade 
progression, mobility, and test score performance. The primary data sources are intended to 
provide detailed information about after school program characteristics and operations. The 


data for the evaluation is summarized in Appendix A. 
Sampling Design 


To address all the research questions stipulated in the evaluation’s Statement of Work 
(SOW), four study samples were constructed based on program participation during the 
2007-08 school year. This section describes the process CRESST used to define each sample. 
An overview of each sample is presented in Table 1. Chapter II will explain the data 


collection process in all of the four samples, while the in-depth analysis of these collected 
data will be conducted in Chapters IV, V, and VI in this report. 


Table 1 


Overview of Study Samples 


Sample Purpose Sampling universe Selection criteria 

Sample I Examine statewide after All schools in the STAR After school participants attending a 
school attendance database with an after school (based on STAR 2007-08) 
patterns and estimate school program with at least 25 after school 
effects of after school participants or at least 25% of all 
participation on students participating in an 
academic achievement ASES/21st CCLC after school 

program 

Sample II Examine behavioral School districts with at Sample of 100 ASES/21st CCLC 
outcomes from district- least one school districts based on probability- 
collected data (e.g., participating in an after proportional-to-size sampling, where 
school day attendance school program (as defined _ size is defined by number of students 
and suspensions) by Sample I) in the district’s STAR records 

Sample III Examine characteristics All agencies receiving after After school agencies and program 
of after school agencies school funding and each of _ sites that returned the After School 
and program sites their program sites Profile Questionnaire and are 

included in Sample I. 

Sample IV In-depth examination of All schools in Sample II Random selection of 40 ASES/21st 
after school program districts with an after CCLC schools from sampling 
operations and school program (as defined universe 
participation by Sample I) 

Sample I 


Sample I is intended to include all of the school sites participating in an ASES/21st 


CCLC after school program and are included in the STAR database. The primary purpose of 
Sample I is to examine statewide after school attendance patterns and estimate effects of 
participation on academic achievement. In Year 1 of the study, the study team defined after 
school program participants as any student with at least one hour of after school attendance. 
After school program schools are defined as schools in the STAR data with at least 25 
program participants or at least 25% of the school’s students participating in an after school 
program. 


The students’ after school attendance data were then merged with the STAR database 
using the Statewide Student Identifier (SSID), and schools with after school programs were 
identified based on each participant’s CDS code as reported in the STAR 2007-08 data. Since 


the ASES/21st CCLC funding focuses on elementary and middle schools students, Sample I 
is restricted to students in grades 2-8. In both Years 1 and 2, these two inclusion criteria 
resulted in 380,410 after school participants for Sample I (or about 98% of participants found 
in the STAR data). The 380,410 students included in Sample I cover 3,053 schools, 415 
districts, and 54 of the 58 counties in California. 


Creating a Comparison Group for Sample I (Propensity Score Matching). To 
examine the effect of after school participation on measurable outcomes, such as California 
Standards Test (CST) performance or attendance, it is necessary to know not only how 
participants fare on these outcomes, but also how they would have fared if they had not 
participated in an after school program (Holland, 1986; Morgan & Winship, 2007; Rubin, 
2005; Schneider, Carnoy, Kilpatrick, Schmidt, & Shavelson, 2007). In Year 1, a comparison 
sample was established using propensity score matching. The propensity score matching 
approach is a popular strategy in producing unbiased estimates of program effects when one 
can accurately estimate the selection process (Morgan & Winship, 2007; Rosenbaum & 
Rubin, 1983). 


Sample II 


The second evaluation question inquires about the effect of after school participation on 
student behavior-related outcomes. Since student-level, behavior-related outcomes are not 
collected by the state, CRESST drew a probability sample of California districts to gather 
district-maintained student behavior data. The primary behavior data to collect from 
Sample II districts are school attendance/absence data, student suspension data, and student 
course marks (e.g., citizenship and work habits). As stipulated in the statement of work, a 
sample of 100 districts was identified in Year 1 for the ASES study. Of these districts, 91 
submitted all or part of the requested data. As a result, Sample II includes 274,893 after 
school participants from 1,804 schools that span across 26 counties in California. 


Sample III 


In order to provide the CDE with a comprehensive overview of the grantees they serve 
and to monitor the grantees’ annual development and change, CRESST designed a profiling 
data system to collect grantee level information on an annual basis. Beginning 
March 22, 2010, CRESST requested for all agencies and after school program sites on record 
as receiving ASES and/or 21" CCLC funding in 2009-10 to complete an after school profile 
questionnaire via a web-based survey application. The purpose of the questionnaire is to 
examine both grantee- and site-level practices and characteristics. To connect these practices 


and characteristics to student outcomes, Sample III was constructed based on the program 


sites in Sample I that completed the after school profile questionnaire by July 15, 2010. For 
Year 2, Sample III includes 1,073 schools, with 190,760 Sample I after school participants. 
The schools span 172 districts and 43 counties in California. 


Sample IV 


To address the more in-depth research questions about after school program operations 
and activities, CRESST selected a random sample of 40 after school sites to conduct 
observations, interviews, surveys, and focus groups. Sample IV sites were selected at random 
from the 1,812 school sites in Sample II. The resulting sample of 40 sites covers 5,629 
students across 24 districts and 13 counties in California. For the Year 2 Annual Report, 
analysis of Sample IV data is restricted to 21 sites in 6 districts and 5 counties, for which data 
were collected prior to July 15, 2010. Data collection procedures for Sample IV are presented 
in Chapter III. 


Early in this research, CRESST constructed the sampling design so that after school 
participants in each study sample would be representative of all after school participants in 
the state, on average. Representativeness of Sample III, however, is dependent on which 
school sites responded to the profiling questionnaire, and representativeness of Sample IV is 
limited by the small number of schools selected for in-depth analysis. The student 
characteristics come from the 2006-07 and 2007-08 STAR data, and the school 
characteristics come from the CDE Public Schools Database and the 2007 Base Academic 
Performance Index (API) file. 


In the Year 1 Annual Report, the research team documented the characteristics of 
students in Samples I through IV. Students in the samples were representative of the 
population of all after school program attendees in the state, with some expected small 
differences likely due to sampling variations. Since Sample III is dependent on which school 
sites respond to the profiling questionnaire each year, the student and school characteristics 
for Sample III can change from year to year. Tables Al and A2 in Appendix A presents a 
comparison of student and school characteristics for the 2008-09 Sample III and the 2009-10 
Sample III. Overall, the students represented by each year are very similar and reflect the 
broader after school participant population (Sample I). 


As previously mentioned (and discussed in more detail in the following chapter), not all 
the data of the sites originally selected for Sample IV has been included in this report. 
Overall, students in the 23 sites included in the analysis of Sample IV reflect the original 
Sample IV student population, and the broader after school participant population (Sample J). 
Student characteristics of the original and included Sample IV population are presented in 


Table A3 in Appendix A. A few notable differences between the original and included 
Sample IV populations exist. For example, compared to the original Sample IV population, 
the included Sample IV has a slightly higher percentage of Hispanic students (84% vs. 76%) 
and more Title I students (98% vs. 85%). Additionally, the included Sample IV had a slightly 
higher percentage of students in the middle school grades (53% vs. 45%). Data will be 
collected at all 40 sites in Year 3; any variation in sampling will be addressed statistically in 
the final report. 


CHAPTER III: 
DATA COLLECTION IN YEAR 2 


As in Year 1, CRESST received statewide CDE data and collected data from after 
school programs through the CRESST-developed After School Profile Questionnaire in Year 
2. Year 2 also marked the first year CRESST collected additional student-level data from the 
Sample II districts and site-level data from Sample IV sites. This chapter details the data 
collected in Year 2. 


Sample I 


In Year 1, CRESST received statewide data for the school years leading up to and 
including 2007-08. For the same time period, the CDE also provided CRESST with after 
school attendance. To extend the analysis of program participation effects on state 
standardized tests to the 2008-09 school year, the CDE provided CRESST with statewide 
data for 2008-09. The following data sources were included in this update: 

e STAR 

e CELDT 


e Physical Fitness Test 


CRESST is still in the process of receiving 2008-09 after school attendance data and CSIS 
(California School Information Services) data from the CDE. As a result, these data are not 
included in the Year 2 analysis. Table A5 in Appendix A summarizes the current status of the 
data for the evaluation. 


Sample II 


With student-level behavior-related outcomes not collected by the state, CRESST drew 
a probability sample of California districts to address the second evaluation question on the 
effect of after school participation on student behavior-related outcomes. As stipulated in the 
SOW, the study team drew a sample of 100 districts for the ASES/21st CCLC study and a 
sample of 30 districts for the ASSETs study. The resulting sample of 100 ASES/21st CCLC 
districts includes 15 districts also selected for inclusion in the 30 Sample II ASSETs districts. 
The 100 Sample II districts include 263,470 after school participants in 1,812 schools, 
spanning 32 counties in California. The behavior data collected from Sample II districts 
includes school attendance, suspensions, and student course marks (e.g., citizenship and work 
habits). 
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Data collection from 100 Sample II districts for the 2006-07, 2007-08, and 2008-09 
school years began in January 2010. In a group e-mail, the CDE consultants sent a data 
request to superintendents and regional leads. Included in the email was information about 
the evaluation as well as a guide to assist districts in completing the request. The guide 
instructed districts to contact CRESST for technical assistance when completing the request. 
District staff uploaded files to the exFiles File Transfer System created by the CDE, and the 
CDE then provided CRESST with the data to process, clean, and analyze. Of the 100 districts 
from which data were requested, 91 districts provided data. Six districts had not provided 
data at the time this report was written despite several contact attempts. Three districts 


received permission not to complete the data request due to special circumstances. 


Of the Sample II districts that provided data, 89% (n = 81) gave complete data (i.e. 
usable, error free data for the 2006-07, 2007-08, 2008-09 school years). A slightly fewer 
number, 86% (n = 78) provided CRESST with complete suspension data. Districts had the 
greatest difficultly with providing behavior course marks; less than half of the 100 districts 
gave CRESST complete behavioral data (n = 42). Barriers to data collection as cited by 
districts in the drawn sample included inconsistent reporting by school sites to the district, a 
lack of electronic recordkeeping by districts, and a lack of appropriately trained staff to 
compile the data request. 


To investigate whether the Sample II data received were representative of the students 
in the STAR population, Sample I, and the drawn Sample II, student and school 
characteristics variables were compared for these groups of students. Tables A6 and A7 in 
Appendix A present the student and school characteristics by sample type. As shown, student 
and school characteristics of the Sample II data were very similar to student and school 
characteristics of the drawn Sample II, Sample I, and the STAR population. 


Sample II 


Based on the grantee list provided by the CDE, CRESST designed a database system to 
house all individual after school profiles* created for the study. The CRESST-designed after 
school profile questionnaire was sent to all after school programs annually to gather in-depth 


information about their program goals, structures, and process. In order to gather information 


* School profiles are profiles created by CRESST for each after school program grantee. The profiles contains 
information like location of the school, number of sites the after school program operates with, the number of 
students served, program goals, how the program is structured (local partnerships, scheduling, activities offered, 
etc.), what program process the after school program employs (relationships, motivational strategies, 
management styles, etc.), and what outcomes are measured. The profiles will be updated each year with the 
Profile and Performance Information Collection System (PPICs) from Learning Point Associates (LPA) and the 
CDE data. 
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from different perspectives, the questionnaire was developed with two sections. Part A of the 
questionnaire focused on the grantee perspective by surveying the program directors. In 
contrast, the Part B questionnaire focused on the site perspective by surveying the site 


coordinators. 


The data from the questionnaire were entered into their individual profiles and updated 
each study year. At the end of Year 3, these programs will be sorted by their program 
characteristics, such as program goals, content focus, organizational structures, program 
delivery style, etc. Then programs will be categorized into different program types and styles 
such as tutoring programs, programs that focused on specific content (e.g., art or science), 
school district affiliated programs, community-based programs, and so on, for further in- 


depth analyses. CRESST refers to this sample as Study Sample III. 
Data Collection Process 


In order to obtain an optimal level of response, several dissemination strategies were 
researched by CRESST. After careful testing and consideration, a web-based data collection 
system was selected. To further promote the response rate and to ensure that the web links to 
the questionnaires reached the intended participants at both the grantee and site levels, 
CRESST conducted a thorough review of the contact list provided by the CDE. This review 
was done by calling and/or emailing the contacts on record for the grants and asking them to 
verify or update the program director and site information. Contact was also made with the 


regional leads in order to update the program director information. 


During Year 2, data collection for Part A of the questionnaire was initiated after the 
majority of contacts were verified (March 22, 2010). Subsequent waves of data collection 
were conducted on an ongoing basis as additional contacts were verified. The web links for 
Part A of the questionnaire were sent directly to the program directors. Within five business 
days of their participation, program directors were emailed to request the current contact 
information for the site coordinators under his/her authority. The web links for Part B of the 
questionnaire was initiated upon receipt of the verified site coordinator contacts 
(April 1, 2010). 


CRESST regularly monitored the completion of the questionnaires, sending reminder 
notices as appropriate to the program directors and site coordinators. Data collection for the 
questionnaires was formally closed on July 15, 2010, after which data cleaning commenced. 
Issues that were handled by CRESST during this process included the handling of 
inconsistencies (or missing responses) concerning the grantee names, site names, and/or CDS 


codes. This resulted in 306 program directors completing Part A, for a response rate of 77.3% 
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during Year 2. In addition, 1,336 site coordinators completed Part B of the questionnaire.” 
The number of responses represent an increase in participation by the program directors, as 
well as a decrease in participation by site coordinators from the Year 1 data collection 
(Part A = 269, Part B = 1,888). Possible reasons for the lower response rate on the Part B 
survey include staff turnover at sites during the Year 2 data collection, confusion that some 
site coordinators expressed concerning research requests they were receiving by other 


studies, as well as greater job pressures because of the economy. 
Sample IV 


In order to gather site-based evidence concerning the theoretical model, a subsample of 
schools (40 for ASES and 21° CCLC) were drawn using stratified random sampling 
methods. This ensures that the selected schools represent the entire population of all schools 
with ASES- and/or 21° CCLC-funded after school programs and that the findings for this 
subsample can be broadly generalized. CRESST refers to this sample, on which much more 
detailed data will be gathered, as Study Sample IV. Observations, surveys, interviews, and 
focus groups were designed by CRESST with input from the CDE and after school 
community. The first cycle of data collection took place from the winter to the summer of 
2010. 


Data Collection Process 


The CRESST research team employed both qualitative and quantitative research 
methodologies, including interviews, focus groups, surveys, and program observations for 
the investigation of Sample IV sites. After obtaining permission from the district IRBs, the 
team contacted the school principals asking them to participate in the study and to provide a 
letter of compliance. Once this was done, the team proceeded with the remaining 


methodologies, as described in the following paragraphs. 


Adult surveys. Site coordinators, site staff, and parents were each surveyed once 
during the school year. CRESST mailed or hand-delivered the surveys to the sites along with 
the information sheets used for consent. The instruments were completed at a time 
convenient for the participants and returned to the CRESST researchers at the time of the site 
visits. Site coordinator and site staff surveys each asked questions about program satisfaction, 
program process, and community partnerships. Site coordinator surveys also asked questions 
about program goals. Parent surveys also asked questions about program satisfaction and 


process, as well as participation in the program. 


Since a small percentage of the after school sites have multiple programs and/or are funded by multiple 
grantees, the total response rate for the site coordinators is not presented. 
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Student surveys. CRESST sent parent permission forms to the site coordinators for 
distribution to the parents of students who participate in the program. CRESST staff 
distributed the student assent forms and administered the student surveys at the time of the 
site visits. Students were asked questions about program satisfaction, program process, their 
participation in the program, and the impact of the program on their learning and 


development. 


Principal and project director interviews. CRESST conducted interviews with 
school principals and project directors from the after school programs. The consent forms 
were hand delivered or sent electronically to the principals and project directors. Once the 
consent forms were signed and returned, their interviews were conducted by telephone or in 
person. Each of these interviews lasted 30-60 minutes and were audio taped. The project 
directors interviews included questions on program satisfaction, program goals, program 
process, community partnerships, and unintended consequences of program participation. 
Similarly, the principal interviews included questions on program satisfaction and unintended 


consequences. 


Staff focus group. CRESST conducted focus groups with site staff at the time of their 
site visit. Site staff were asked to sign a consent form prior to the start of the focus group, 
which generally lasted 30 to 60 minutes. Focus group protocols for use with site staff 


included questions on program satisfaction, program process, and community partnership. 


Student focus group. CRESST sent parent permission forms to the site coordinators 
for distribution. CRESST staff distributed the student assent forms and conducted the focus 
groups at the time of their site visits. One or two focus groups were conducted per site, each 
consisting of about four to six students. These focus groups lasted about 30 to 60 minutes 
each and included questions about program satisfaction, program process, their participation 


in the program, and the impact of the program on their learning and development. 


Observations. Observations were conducted at each of the after school sites. After 
coordinating with the site coordinators, the CRESST researchers observed two to four 
activities at each site with the goal of seeing the major programmatic features. Observation 
checklists and rating sheets focused on program structure and implementation at the after 


school programs. In addition, researchers took field notes to provide additional evidence. 
Recruitment of Participants 


Qualitative and quantitative research methodologies were employed at after school sites 
that were funded by the ASES and/or 21° CCLC programs. This included 16 elementary 


schools and 5 middle schools, representing 6 districts. All recruitment of sites was conducted 
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by the research staff, and permission was obtained from the districts and school principals to 
conduct surveys, focus groups, interviews, and observations. The after school programs 
helped the research staff to distribute and collect the site coordinator surveys, site staff 
surveys, parent surveys, and parent permission forms. It should be noted that the research 
team failed to meet the goal of recruiting 40 sites during the first phase of the study. This was 
the result of difficulties in obtaining permission from some of the districts and schools that 
were initially selected for participation. Table 2 shows the specific number of participants 


who were recruited for the surveys, interviews, and focus groups. 


Table 2 
Study Participants by Role 


Interviews and focus 


Participants Surveys groups 

Site staff 

Program directors -- 13 

Site coordinators 18 -- 

Site staff 119 69 
Other Stakeholders 

Principals -- 17 

Students 574 192 

Parents 901 -- 
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CHAPTER IV: 
PRELIMINARY ESTIMATES OF AFTER SCHOOL PARTICIPATION EFFECTS 


This chapter presents preliminary findings on the academic and behavioral outcomes 
for Study Samples I and II. For the Year 1 Annual Report, CRESST examined the effect of 
after school participation on academic achievement during the 2007-08 school year. For the 
matched samples, the research team found that after school participation at the elementary 
level (grades 3-5) had a slightly negative, but substantively weak, effect on participants’ 
English language arts (ELA) and math CST scores. The difference between after school 
participants and the comparison groups was equal to about one scale score point. At the 
middle school level (grades 6-8), no statistically significant difference was found between 
participants and the comparison students. Similarly for the CELDT, a slight negative, but 
substantively weak, effect was found for elementary (grades 3-5) participants, and no 
statistical difference in CEDLT performance was found between middle school (grade 6-8) 


participants and the comparison students. 


For this Year 2 Annual Report, the analysis of after school participation effects is 
expanded in three ways. First, CRESST extended the analysis to include students’ 2008-09 
academic achievement outcomes measures. Second, the research team examined a set of 
behavioral outcomes in addition to the academic achievement outcomes. The behavioral 
outcome variables include performance on the state physical fitness exam, school day 
attendance rate, and school suspension rate. Please refer to Table 3 for the specific academic 
achievement and behavior outcomes examined in this report. Third, in addition to looking at 
the outcomes for all after school participants (i.e., students with at least one day of after 
school attendance), the team compared the outcomes for a subset of after school participants 
who “frequently” attended an after school program. The definition and description of 


frequent after school participants will be discussed in the next section in this chapter. 
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Table 3 


Outcomes Examined for the Year Two Annual Report 


Data coverage 


Outcomes Sample 2007-08 2008-09 


Academic achievement outcomes 


ELA CST scale score I v v 

Mathematics CST scale score I v v 

CELDT overall scale score I v v 
Behavior outcomes 

Fitnessgram® healthy fitness zone attainment I v 

School day attendance rate II v v 

Ever suspended from school II a “a 


Note. CELDT = California English Language Development Test; CST = California Standards Tests; 
ELA = English language arts. 


Since the 2008-09 after school participation data was not available in time for inclusion 
in this report, CRESST will analyze the data for the later report to determine whether 2007- 
08 participants continued in an after school program or whether 2007-08 control students 
entered an after school program in 2008-09. Additionally, the propensity score matching used 
to create comparable participant and control groups was based on the 2007-08 data; 
therefore, CRESST will also examine later how group comparability degrades over time due 
to students switching program participation or across different outcome measures. As a 
result, findings for the 2008-09 outcomes should be interpreted with caution and should be 
considered preliminary. Future analyses will investigate how the matched group 


comparability changes over time and across outcome measures. 


The rest of this chapter consists of a section on after school participation and frequent 
participants, a section on the findings for the analysis of academic achievement outcomes and 
behavior outcomes for Sample I, and a parallel section for the analysis of behavior outcomes 
for Sample II.* Throughout this chapter, academic achievement and behavior outcomes are 
reported for students in the propensity score matched sample with non-missing data for the 


relevant outcome measures. 


“Sample I includes all schools in the STAR database with an after school program, and this is the data used for 
examining statewide after school attendance patterns and estimating the effects of after school participation on 
participants’ academic achievement. Sample II includes a sample of 100 ASES/21st CCLC and its purpose is 
for examining behavioral and achievement outcomes. 
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Levels of After School Participation and Frequent Participants 


The state funding through ASES mandates that school sites stay open until 6:00 p.m. 
with programs running a minimum of fifteen hours per week. In addition, all participating 
elementary students must attend the program five days a week, Monday through Friday. 
Middle school students are required to attend the program three consistent days per cycle. 
With that said, students had varying levels of attendance, and not all program participants 
attended as many days as required. As shown in Figure 2, elementary students were more 
likely to attend relatively regularly, while middle school students tend to attend less 
consistently (26% attended less than 10 days). 


After School Program Attendance 


@ Elementary Attendance (N=281,054) Middle School Attendance (N=146,292) 
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Figure 2. Distribution of after school program attendance for elementary and middle school students, 2007-08. 
A small number of students have more than 180 days of after school attendance. Generally, these numbers 
represent students that have transferred schools during the year and attend a school with a different calendar. 


In consideration of the varying participation requirement for elementary and 
middle/high school students, different cutoff days were used to classify participants in terms 
of whether they were frequent participants or not. CRESST defined frequent participants as 
students that attend after school programming the majority of days required. At the 
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elementary level, frequent participants were those students who attended the program three 
days a week, or 108 days or more during the school year. With the above definition, frequent 
participants represent 52% of elementary after school attendees. In middle school, students 
were classified as frequent participants if they attended at least 72 days during the school 
year, equivalent to two or more days a week. Thirty-two percent of middle school students 
participated at least two days a week. 


Tables B1 and B2 in Appendix B report the student characteristics and school 
characteristics for frequent participants, all participants, and non-participants for Sample I 
and the matched sample”. A review of the student characteristics of frequent participants, all 
participants, and non-participants for the Sample I and matched sample data shows that all 
groups are similar demographically, and frequent participants in both samples have nearly 
identical student and school characteristics as the control and the overall participant groups, 
with a few exceptions. Slight differences between CST scores were found in the matched 
sample; frequent participants were on average more likely to have higher CST math and ELA 
scores. Frequent participants were also less likely to be new to their school during the 
2007-08 school year and were more likely to come from smaller schools that had slightly 
higher API scores. 


Academic Achievement Outcomes (Sample I) 


There are two academic achievement outcome measures: CST and CELDT. As in the 
Year 1 Annual Report, CST and CELDT scale scores were standardized based on the 
statewide mean and standard deviation for each subject test. Standardization puts the scale 
scores on a common metric and aids comparability across grades, tests, and years. A 
standardized scale score of zero means the student scored at the mean for all other students in 
the state who took the same test. A standardized scale score of 1.0 means the student scored 
one standard deviation higher than the statewide mean, and a standardized scale score of -1.0 


means the student scored one standard deviation lower than the statewide mean. 
Performance on the CST 


To examine the effect of after school participation on participants’ academic 
performance, the research team examined their performance on the CST ELA and 
mathematics. Table 4 reports the mean performance on the ELA and mathematics CST for 
three groups of students: (1) the matched control group (students who didn’t participate in the 


° Non-participants are only included in the demographic analysis to examine the representativeness of the 
different student categories. Non-participants will be excluded from any future analysis; the matched 
comparison group will be used instead. 
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after school program in 2007-08 and were randomly matched to the after school participants), 
(2) 2007-08 after school participants (ASPs), and (3) 2007-08 ASPs who were classified as 
frequent participants. The mean scores are relatively steady across the two years and similar 
between the groups. For example, among the elementary schools students (grades 3-5), all 
three groups of students scored about 0.3 standard deviation below the statewide mean in 
ELA and about 0.2 standard deviation below the statewide mean in mathematics for both 
2007-08 and 2008-09. With that said, frequent ASPs were found to perform slightly better 
than both the control group and the overall ASP group for both years and for both ELA and 
mathematics. 


Table 4 


Standardized CST Scale Score Means for Matched Control and 
After School Participant Groups 


Outcome Control ASP Freq. 


Elementary school grades 


ELA CST, 2007-08 -0.299 -0.340 = -0.286 
ELA CST, 2008-09 -0.295 -0.332 = -0.284 
Math CST, 2007-08 -0.228 -0.242 -0.177 
Math CST, 2008-09 -0.236 -0.254 = -0.195 
Middle school grades 
ELA CST, 2007-08 -0.289 -0.283 = -0.236 
ELA CST, 2008-09 -0.278 -0.275 = -0.231 
Math CST, 2007-08 -0.154 -0.127  ~—_-0.100 
Math CST, 2008-09 -0.100 -0.084 = -0.041 


Note. ASP = After school participant; CST = California Standards 
Test; ELA = English language arts; Freq = Frequent after school 
participant. 


Since some residual differences in ability might exist between the control and 
participant groups—even after incorporating the propensity score methodology discussed in 
the Year 1 report—the research team further estimated the participation effect by adjusting 
for each student’s prior CST performance through a regression model (i.e., for the ELA CST 
outcomes the team controlled for 2006-07 ELA CST scores, and for the mathematics CST 
outcomes the team controlled for 2006-07 mathematics CST scores). The regression-based 
estimates are reported in Table 5. 
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Table 5 
Estimated Effect of After School Participation on Standardized CST Scale Score (Matched Sample) 


All ASP Frequent ASP 
Outcome N Estimate (SE) N Estimate (SE) 

Elementary school grades 

ELA CST, 2007-08 247,508 -0.030 (0.003) ** 169,025 -0.017 (0.003) ** 

ELA CST, 2008-09 242,487 -0.032 (0.003) ** 165,772 -0.021 (0.004) ** 

Math CST, 2007-08 248,820 -0.006 (0.003) 169,862 0.021 (0.004) ** 

Math CST, 2008-09 242,775 -0.015 (0.004) ** 165,988 0.009 (0.005) * 
Middle school grades 

ELA CST, 2007-08 217,163 -0.003 (0.006) 117,744 0.020 (0.007) ** 

ELA CST, 2008-09 203,197 -0.001 (0.006) 110,012 0.018 (0.008) * 

Math CST, 2007-08 216,195 0.012 (0.006) 117,214 0.020 (0.011) 

Math CST, 2008-09 203,972 0.009 (0.007) 110,420 0.034 (0.011) ** 


Note. Participation effect estimates control for 2006-07 CST scores. Standard errors adjusted for school-level 
clustering. ASP = After school participant; CST = California Standards Test; ELA = English language arts. 
* p-value < 0.05; ** p-value < 0.01 


Among elementary grade students (grades 3-5), the overall effect of after school 
participation on ELA CST performance was slightly negative, but substantively weak. For 
example, on the 2007-08 ELA CST, ASPs scored, on average, 0.03 standard deviations lower 
than the control group, and frequent ASPs scored, on average, 0.017 standard deviations 
lower than the control group (after adjusted for prior year ELA CST performance). While 
this difference meets traditional thresholds for statistical significance, it only represents a 
difference of about one or two scale score points. The after school participation effect was 
smaller for the mathematics CST and slightly positive among frequent ASP. Among middle 
school students (grades 6-8), after school participation had no statistically significant effect 
on ELA or mathematics CST performance, on average, for all ASPs. For frequent ASPs, the 


effect was slightly positive but, again, represented a substantively weak effect size. 
Performance on the CELDT 


English learners (ELs) account for 42% of the Sample I ASPs. To assess the English 
language development of EL students, California requires EL students to take the CELDT 
each year. Students’ overall CELDT scores are used to see the effect of after school 
participation on ELs’ English language development. Average performance on the CELDT is 
reported in Table 6. Mean standardized scale scores are reported separately for the matched 
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control and ASP groups, as well as for frequent participants. For all groups, the mean scores 
are much higher in 2008-09 than the scores for 2007-08. This increase likely reflects the 
language development that occurs over the year, since the students are a year older in 
2008-09. It is important to note that the analysis for this report focuses on the within-year 
comparison of ASP and control students and was not designed to examine growth of time. 
Within each year, the average Overall CELDT scores are very similar across groups, with the 
average scores for frequent ASPs slightly higher than both the control group and the overall 
ASP group. 


Table 6 


Standardized Overall CELDT Scale Score Means for Matched 
Control and After School Participant Groups 


Outcome Control ASP Freq. 


Elementary school grades 


Overall CELDT, 2007-08 0.035 0.018 0.044 

Overall CELDT, 2008-09 0.197 0.186 0.221 
Middle school grades 

Overall CELDT, 2007-08 0.043 0.043 0.051 

Overall CELDT, 2008-09 0.104 0.116 0.174 


Note. ASP = After school participant; CELDT = California English 
Language Development Test; Freq = Frequent after school participant. 


As with the analysis of CST performance, the CRESST team estimated the 
participation effect on CELDT performance with a regression model that adjusts for each 
student’s 2006-07 CELDT performance to account for potential residual differences in ability 
that might exist between the control and participant groups, even after incorporating the 
propensity score methodology. The regression-based estimates are reported in Table 7. 
Among elementary grade students (grades 3-5), after school participation had no statistically 
significant effect on EL students’ overall CELDT performance. Frequent after school 
participation, however, was associated with a minor statistically significant positive effect of 
0.021 standard deviations on the 2008-09 CELDT, as compared to the matched control 
group. The same pattern held for middle school students (grades 6-8), where the only 
Statistically significant effect was on the 2008-09 CELDT. Again, however, this effect was a 
substantively minor effect of 0.063 standard deviations. 
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Table 7 
Estimated Effect of After School Participation on Standardized Overall CELDT Scale Scores (Matched Sample) 


All ASP Frequent ASP 
Outcome N Estimate (SE) N Estimate (SE) 

Elementary school grades 

Overall CELDT, 2007-08 112,482 -0.006 (0.006) 75,392 0.003 (0.007) 

Overall CELDT, 2008-09 94,004 -0.004 (0.006) 62,263 0.021 (0.007) ** 
Middle school grades 

Overall CELDT, 2007-08 61,717 0.015 (0.012) 33,119 0.003 (0.014) 

Overall CELDT, 2008-09 51,458 0.018 (0.010) 27,908 0.063 (0.013) ** 


Note. Participation effect estimates control for 2006-07 CELDT overall scores. Standard errors adjusted for 
school-level clustering. ASP = After school participant. CELDT = California English Language Development 
Test. 

* p-value < 0.05; ** p-value < 0.01 


Behavior Outcomes 


Most after school programs aim to affect more than just student academic achievement. 
While the Year 1 Annual Report focused on academic outcomes, this annual report has been 
extended to examine three behavior outcomes: physical fitness, school day attendance, and 
school suspensions. The physical fitness analysis covered Sample I students by using the 
statewide data from the Fitnessgram® assessment. The analysis of attendance and 
suspensions was restricted to students in the districts from which CRESST requested and 
received the appropriate data (Sample II), as the individual-level school day attendance and 
school suspension data are not collected by the state. For the future reports, CRESST plans to 
include an analysis of two additional behavior outcomes: school mobility and classroom 


behavior (e.g., citizenship). 
Physical Fitness (Sample I) 


Student health has become an increasing concern among schools over the past decade, 
and most ASES and 21* CCLC after school programs include a recreational or physical 
fitness component. To examine whether after school participation benefits student health, the 
research team analyzed student performance on the 2008-09 Fitnessgram® assessment 
(2007-08 data were not available). Fitnessgram® is a physical fitness assessment program 
administered by the state to students in grades 5, 7, and 9. The assessment program includes 
a variety of health-related physical fitness tests designed to assess cardiovascular fitness, 


body composition, muscle strength, muscular endurance, and flexibility. Based on criterion- 
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referenced health standards, Fitnessgram® tests students in six fitness categories and reports 
whether the student falls into the “healthy fitness zones” (HFZ) for each category (Welk & 
Meredith, 2008). 


For this report, the research team looked at the percentage of students who met the HFZ 
criteria for each of the six fitness categories and tested whether after school participation 
increased the likelihood of falling into an HFZ. CRESST estimated the effect of after school 
participation on the likelihood of attaining an HFZ with a separate logistic regression model 
for each fitness category, controlling for their 2006-07 ELA CST scale score to improve 
group comparability. Since after school program participation is defined in 2007-08, and 
since CRESST has fitness data for grades 5, 7, and 9 in 2008-09 only, results for the 
elementary school students only reflect participants who were in 4" grade in 2007-08, and 
results for the middle school student participants who were in sixth and eighth grade in 2007- 
08. 


The percent of students in each of HFZ is reported in Table 8 About 60% to 85% of the 
students fall within the HFZ depending on the fitness category. More students attained the 
trunk strength HFZ (over 80%) than any other fitness categories, while fewer students 
attained the aerobic capacity HFZ (less than 65%) than any other categories. ASPs tended to 
perform slightly better on the Fitnessgram®, on average, than control students, and frequent 
participants performed better than regular ASPs. For example, 60.3% of elementary school 
control group students met the aerobic capacity HFZ, compared to 62.9% of the elementary 
school ASPs and 64.7% of the elementary school frequent ASPs. 
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Table 8 


Percent of Students in Healthy Fitness Zones (2008-09) for Matched 
Control and After School Participant Groups 


Outcome Control ASP Freq. 


Elementary school grades 


Aerobic capacity 60.3% 62.9% 64.7% 
Body composition 62.0% 62.9% 64.1% 
Abdominal strength 76.6% 77.7% 78.9% 
Trunk strength 86.0% 86.8% 88.0% 
Upper body strength 64.0% 65.4% 66.5% 
Flexibility 65.2% 66.0% 67.1% 
Middle school grades 
Aerobic capacity 57.7% 61.9% 64.1% 
Body composition 61.7% 63.0% 64.7% 
Abdominal strength 79.4% 81.0% 83.0% 
Trunk strength 86.1% 87.4% 88.4% 
Upper body strength 67.8% 69.3% 70.6% 
Flexibility 73.8% 75.3% 75.4% 


Note. ASP = After school participant; Freq = Frequent after school 
participant. 


The estimated effects of after school participation on the percent change in the 
likelihood of meeting the HFZ benchmarks are reported in Table 9. For both elementary 
school and middle school students, being an ASP or a frequent ASP was associated with a 
statistically significant increase in the likelihood of meeting each HFZ benchmark (with the 
exception of flexibility for elementary school students). The largest estimated effect was for 
aerobic capacity. For elementary school, ASPs had an 11.5% increase in the likelihood of 
meeting the aerobic capacity HFZ compared to control students, and frequent ASPs had a 
19.9% increase in the likelihood compared to control students. For the middle school grades, 
the estimated effects were slightly higher. ASPs had an 18.5% increase, and frequent ASPs 
had a 29.4% increase in the likelihood of meeting the aerobic capacity HFZ, compared to 


control students. 
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Table 9 


Estimated Effect of After School Participation on 2008-09 Healthy Fitness Zone Attainment (Matched 
Sample) 


All ASP Frequent ASP 
Outcome N Estimate (SE) N Estimate (SE) 

Elementary school grades 

Aerobic capacity 81,767 11.5% (1.8%) ** 56,059 19.9% (2.2%) es 

Body composition 81,767 4.0% (1.8%) * 56,059 8.8% (2.1%) mm 

Abdominal strength 81,767 6.7% (2.1%) ** 56,059 13.1% (2.6%) ee 

Trunk strength 81,767 6.7% (2.6%) * 56,059 18.5% (3.6%) ake 

Upper body strength 81,767 6.0% (1.9%) ** 56,059 10.5% (2.3%) iid 

Flexibility 81,767 3.5% (1.9%) 56,059 7.3% (2.2%)  ** 
Middle school grades 

Aerobic capacity 136,285 18.5% (2.1%) ** 74,949 29.4% (3.1%)  ** 

Body composition 136,285 5.0% (1.8%)  ** 74,949 12.1% (2.4%)  ** 

Abdominal strength 136,285 10.1% (2.6%)  ** 74,949 24.9% (3.8%)  ** 

Trunk strength 136,285 11.4% (3.0%)  ** 74,949 20.9% (3.8%)  ** 

Upper body strength 136,285 6.6% (2.1%) ** 74,949 12.3% (2.7%) om 

Flexibility 136,285 76% (2.4%)  ** 74,949 6.9% (2.9%) * 


Note. Participation effect estimates reflect percent change in likelihood of attaining healthy fitness zone, using 
a logistic regression controlling for 2006-07 ELA CST score. Standard errors adjusted for school-level 
clustering. ASP = After school participant. 

* p-value < 0.05; ** p-value < 0.01 


School Day Attendance (Sample IT) 


School day attendance can be both a reflection of school engagement and a necessary 
intermediary for student learning. To examine whether after school participation improves 
day school attendance, CRESST requested student-level school attendance data from all 
Sample II school districts. The attendance data were converted into attendance rate for a 
given school year based on the number of days enrolled and days absent that were reported 
by each Sample II district.® Average attendance rates for the 2007-08 and 2008-09 school 


° All data provided by Sample II districts were reviewed and cleaned by CRESST. Attempts were made to 
contact districts that provided data with large amounts of student records showing over 180 days enrolled. In 
cases in which districts did not respond or were unable to provide revised data, cases with over 200 days 
enrolled were not included in the analysis. In many districts, duplicate cases for students were found. Using the 
assumption that students attended more than one school in a year, attendance values for duplicate cases were 
added together if the total number of days enrolled was less than 200 days. In instances where duplicate cases 
were greater than 200, the case with the highest number of days enrolled was included in the analysis. 
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years are reported in Table 10 by schooling level and group. In general, the average 
attendance rates were above 95% and did not differ much across years or groups. Attendance 
rates are slightly higher for elementary grade students compared to middle school students, 
and slightly higher for frequent ASPs compared to the overall ASPs and control students. 


Table 10 


School Day Attendance Rate Means for Matched Control and After School 
Participant Groups 


Outcome Control ASP Freq. 


Elementary school grades 


Attendance rate, 2007-08 95.9% 96.7% 97.1% 

Attendance rate, 2008-09 96.1% 96.7% 97.0% 
Middle school grades 

Attendance rate, 2007-08 95.1% 96.2% 96.8% 

Attendance rate, 2008-09 94.5% 95.4% 96.0% 


Note. ASP = After school participant; Freq = Frequent after school 
participant. 


The research team estimated the effect of after school participation on the school day 
attendance rate with a regression model that adjusts for both a student’s 2006-07 attendance 
rate and 2006-07 ELA CST scale score. The regression-based estimates are reported in 
Table 11. For both elementary and middle school grades, after school participation had a 
Statistically significant but substantively minor effect on attendance rates. For example, for 
elementary grade students, attending an after school program was associated with a 0.4 
percentage-point increase in a student’s attendance rate in 2007-08, which roughly amounts 
to less than one additional school day, on average. The estimate effect for frequent ASPs was 
slightly higher, with an estimated effect of 1.1 percentage points in 2007-08 for frequent 
ASPs in the middle school grades. One should note that for the analysis of frequent ASPs, 
there is ambiguity about whether attending more after school program days results in higher 
school day attendance or whether having higher school day attendance allows one to attend 


more after school program days. 
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Table 11 
Estimated Effect of After School Participation on School Day Attendance Rates (Matched Sample) 


All ASP Frequent ASP 


Outcome N Estimate (SE) N Estimate (SE) 


Elementary school grades 
Attendance rate, 2007-08 85,594 0.004 (0.000) ** 56,008 0.006 (0.000) ** 
Attendance rate, 2008-09 80,961 0.003 (0.000) ** 52,976 0.005 (0.001) ** 
Middle school grades 
Attendance rate, 2007-08 67,265 0.005 (0.001) ** 37,141 0.011 (0.001) ** 
Attendance rate, 2008-09 57,077 0.006 (0.001) ** 31,627 0.012 (0.002) ** 


Note. Participation effect estimates control for 2006-07 attendance rate and ELA CST score. Standard errors 
adjusted for school-level clustering. ASP = After school participant. 
* p-value < 0.05; ** p-value < 0.01 


School Suspensions (Sample IT) 


After school programs seeking to improve positive youth development are expected to 
reduce student behavior problems at school, like school suspensions. To examine whether 
after school participation did reduce school suspensions, CRESST requested student-level 
school suspension data from all Sample II school districts. In this report, the research team 
analyzed the suspension data in terms of whether a student was ever suspended during a 


given school year. 


The percent of Sample II students ever suspended during the 2007-08 and 2008-09 
school years is reported in Table 12 by schooling level and group. School suspensions are 
more prevalent in the middle school grades than the elementary school grades; about 5% of 
elementary students had been suspended during the 2007-08 school year, and over 10% of 
middle school students had been suspended during that year. Similarly, for the matched 
sample, suspensions were slightly more prevalent in 2008-09 than 2007-08, which most 
likely reflects the fact that suspensions increase during the adolescence years, and students in 
the matched sample are a year older in 2008-09. Within a given grade level and year, the 


percent of suspended students was similar between the three groups. 
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Table 12 


Percent of Students Ever Suspended during the School Year for 
Matched Control and After School Participant Groups 


Mean outcome values 


Outcome Control ASP Freq. 


Elementary school grades 


Ever suspended, 2007-08 4.8% 5.0% 4.2% 

Ever suspended, 2008-09 6.9% 7.4% 6.4% 
Middle school grades 

Ever suspended, 2007-08 13.0% 12.9% 11.7% 

Ever suspended, 2008-09 14.4% 14.2% 13.6% 


Note. Participation effect estimates reflect mean difference from 
control group. P-values based on logistic regression controlling for 
2006-07 ELA CST score and adjusted for school-level clustering. 
ASP = After school participant; Freq = Frequent after school 
participant. 


The research team estimated the effect of after school participation on the likelihood 
that a student was suspended in a given year using a logistic regression model that adjusts for 
both whether a student was suspended in 2006-07 and the student’s 2006-07 ELA CST scale 
score. The regression-based estimates are reported in Table 13. Overall, being an ASP ora 
frequent ASP does not result in a statistically significant change in a student’s likelihood of 


suspension for either elementary or middle school students. 


Table 13 
Estimated Effect of After School Participation on School Suspension (Matched Sample) 


All ASP Frequent ASP 
Outcome N Estimate (SE) N Estimate (SE) 

Elementary school grades 

Ever suspended, 2007-08 98,615 4.6% (5.0%) 64,160 -5.9% (6.1%) 

Ever suspended, 2008-09 91,159 8.0% (4.1%) 59,334 -2.9% (5.1%) 
Middle school grades 

Ever suspended, 2007-08 82,558 5.1% (5.8%) 44,398  -10.0% (7.5%) 

Ever suspended, 2008-09 72,468 4.1% (5.9%) 38,561 -5.9% (7.0%) 


Note. Participation effect estimates reflect percent change in likelihood of being suspended during the school 
year using a logistic regression controlling for whether a student was suspended in 2006-07 and 2006-07. 
Standard errors adjusted for school-level clustering. ASP = After school participant. 
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Summary 


In this chapter, preliminary estimates of after school participation effects on students’ 
academic achievement and behavioral outcomes were presented. With 2008-09 after school 
participation data not available for inclusion in this report, findings reported here for the 
2008-09 outcomes using 2007-08 after school program participation data should be 
interpreted with caution and should be considered preliminary. Future analyses will 
investigate whether 2007-08 participants continued in an after school program, or whether 
2007-08 control students entered an after school program in 2008-09, and how the matched 
group comparability changes over time due to students switching program participation or 
across different outcome measures. More sophisticated analyses drawing on Hierarchal 
Linear Modeling (HLM) and using the site/school as a unit of analysis are planned for next 
year. 


Based on the 2007-08 after school participation data and 2008-09 outcome data, 
CRESST found that after school participation at both elementary and middle school levels 
had slightly negative to no effect on students’ academic achievement outcomes and small 
positive effects on some behavioral outcomes. The most consistent finding was a small, 
Statistically significant, positive effect of after school participation on physical fitness and a 
statistically significant, yet minor, positive effect on school day attendance rates. Please refer 
to Figures 3 and Figure 4 for more detailed results for elementary grade students and middle 
school students, respectively. 


Figures 3 and Figure 4 also report the parallel findings for the frequent participants. For 
most outcomes, the after school program effects were slightly larger for students who 
frequently attended an after school program, rather than just attending at some time during 
the year. Compared to the results found for all after school participants, frequent participants 
had higher mathematics CST scores for both 2007-08 and 2008-09, higher CELDT scores in 
2008-09, and larger positive effects on three physical fitness measures. 


In the coming year, CRESST will extend the analyses to incorporate additional 
behavioral outcomes—student mobility and classroom behavior—and examine variation in 
after school participation effects across schools. Additionally, CRESST will look at the 
longitudinal nature of after school participation and participation effects for specific student 
cohorts. 
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Figure 3. Summary of after school participation effects for elementary grade students. No Stat. Sig. 

Effect = Estimated effect of after school participation was not statistically significant at a 95% confidence level. 
Minor Effect = Estimated effect of after school participation was statistically significant, but the magnitude of 
the effect was weak. For continuous outcomes, minor effects were less than 0.10 of the outcome’s standard 
deviation. For dichotomous outcomes, minor effects were less than a 10% change in the likelihood of obtaining 
the outcome. Small Effect = Estimated effect of after school participation was statistically significant and may 
be substantively meaningful. For continuous outcomes, small effects were between 0.10 and 0.30 of the 
outcome’s standard deviation. For dichotomous outcomes, small effects were between a 10% and 30% change 
in the likelihood of obtaining the outcome. 
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Figure 4. Summary of after school participation effects for middle school students. No Stat. Sig. 

Effect = Estimated effect of after school participation was not statistically significant at a 95% confidence level. 
Minor Effect = Estimated effect of after school participation was statistically significant, but the magnitude of 
the effect was not substantively meaningful. For continuous outcomes, minor effects were less than 0.10 of the 
outcome’s standard deviation. For dichotomous outcomes, minor effects were less than a 10% change in the 
likelihood of obtaining the outcome. Small Effect = Estimated effect of after school participation was 
statistically significant and may be substantively meaningful. For continuous outcomes, small effects were 
between 0.10 and 0.30 of the outcome’s standard deviation. For dichotomous outcomes, small effects were 
between a 10% and 30% change in the likelihood of obtaining the outcome. 
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CHAPTER V: 
DESCRIPTIVE RESULTS OF YEAR 2 GRANTEE 
PROFILING REPORTS (STUDY SAMPLE IIT) 


This chapter presents the descriptive Year 2 results on the data collected and analyzed 
for Study Sample III. Prior to the first year of data collection, CRESST conducted reviews on 
the available annual after school accountability reports from the CDE and the existing Profile 
and Performance Information Collection System (PPICs) from Learning Point Associates 
(LPA), followed with an extensive literature review on out-of-school time. Several critical 
components (i.e., program environment, goal-oriented programs, and program orientation) 
emerged from this synthesis of literature and materials, which CRESST incorporated into the 
theoretical model that guided the development of the profiling system. 


In order to obtain an optimal level of response, a web-based data collection system was 
selected. During year 2, data collection took place from March 22 to July 15, 2010. The web 
links for Part A of the questionnaire were sent directly to the program directors, while the 
web links for Part B were sent directly to the site coordinators. The after school profile 
questionnaire included questions covering the following eight themes: (a) funding sources, (b) 
fee scale and enrollment strategies at sites, (c) student recruitment and retention, (d) goals 
and outcomes, (e) programming and activities, (f) staffing, (g) professional development, and 
(h) community partnerships. Figure 5 illustrates the alignment of these themes to the critical 
components extracted from the synthesis of literature. In addition, the letters in the 
parentheses indicate whether the theme was included in Part A and/or Part B of the 


questionnaire. 


After School Profiling System 


Background Information Program Environment Goal-Oriented Program Orientation 


Funding sources Fee scale and Goals and Staffing (A and B) 
(A and B) enrollment (B) outcomes (A) 


Professional 
Student Programming and development (A 
recruitment and activities (B) and B) 
retention (B) 

Community 

Partnerships (B) 


Figure 5. Organization of the After School Profile Questionnaire. 
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In order to provide more clarity to the analyses, we will present the descriptive results 
separately by the critical components and their themes (See Sections I-IV). Results for the 
Part B questionnaire will be further analyzed by the subgroups of urbanicity and grade span. 
Urbanicity is a variable to classify after school sites by their geographic location within a 
city, suburb, or town/rural area.’ Similarly, grade span classifies after school sites by the 
grade levels that their program serves. In this report, the grade spans reported include 
elementary school and middle school. Tables that provide more details of the results are 
included in Appendix A. In addition, the complete findings for the ASES and 21° CCLC 


program are discussed in a separate descriptive report. 
Funding Sources 


The majority of grantees are funded solely by the ASES program (74.0%). In addition, 
5.3% of grantees are funded solely by the 21 CCLC, 6.3% are funded by both the 21°*CCLC 
and ASES, and 14.4% of the grantees receive both K-9 (ASES and/or 21° CCLC) and high 
school (ASSETs) funding. Of these grantees, 312 out of a possible 396 completed the Part A 
questionnaire. This represents a total response rate of 78.8% for grantees that receive funding 
for K-9 programs. In perspective to the funding streams, the distribution of the Part A 
questionnaires completed was similar to the distribution of the funding. For example, the 
majority (71.2%) of the questionnaires were completed by the program directors for the 
ASES only grantees (See Figures 6 and 7). 


’ Urbanicity was derived from a classification system developed by the U.S. Department of Education Institute 
of Education Sciences (See http://nces.ed.gov/ccd/rural_locales.asp for more information). 


34 


21st CCLC only 
5.3% 


K-9 & ASSETs 
14.4% 


21st CCLC & ASES 
6.3% 
ASES only 74.0% 


Figure 6. Overall funding of grantees (n = 396). 
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Figure 7. Participation by type of funding, Part A questionnaire (n = 312). 


At the site level, the Part B questionnaire was completed by 1,336 site coordinators. 
Since a small percentage of the K-9 sites were funded by multiple grantees or hosted multiple 
after school programs, the total response rate is not presented. In terms of the funding 
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streams, the distribution of the Part B questionnaires completed was similar to the 
distribution for Part A. The site coordinators for the ASES only sites completed the majority 
(84.1%) of the questionnaires. Interestingly, more than half of the sites (54.2%) for which a 
Part B questionnaire was completed were located in the cities. Similarly, the distribution of 
sites by grade span was unequal, with 76.1% of the sites being hosted at schools that serve 
elementary students only. 


Fee Scale and Enrollment Strategies at Sites 


Within the Part B questionnaire, site coordinators were asked to report about their 
enrollment capacity. Interestingly, only 49.0% of the respondents reported that they were 
able to enroll all of the students who wished to participate at their site. When examining the 
results by urbanicity, a moderately greater percentage of respondents in the town/rural areas 
(62.7%) reported that they had enough capacity, than did the respondents from the cities 
(46.8%) and suburbs (44.7%). Likewise, greater percentages of respondents whose sites 
spanned middle school only (75.8%) reported that they were able to enroll all interested 
students when compared with the sites that spanned elementary only (40.8%). 


While about half of the site coordinators reported being able to enroll all of the 
interested students at their sites, it is interesting that an even greater number indicated that 
they maintain a waiting list (78.5%). When examining the results by subgroup, the results are 
the reverse of those concerning enrollment capacity. The respondents from the city (83.8%) 
and suburban (81.2%) areas, who reported a lower ability to enroll all interested students, 
were also much more likely than those from town/rural areas (58.1%) to report that they 
maintained a waiting list. Furthermore, the respondents whose sites spanned elementary only 
were less likely than the respondents whose sites spanned middle school only (50.8%) to 
report that they had enough capacity and were more likely to report that they maintained a 
waiting list (86.9%). 


Site coordinators were also asked about the fees they charge and strategies they use for 
enrollment. In regards to this first issue, the majority of site coordinators (87.8%) indicated 
that participation at their sites is free for all students. This is not surprising since both the 
ASES and 21* CCLC programs discourage sites from charging fees in general and prohibit 
them from excluding children because of lack of finances. The majority of respondents 
(61.6%) also reported that they enroll students on a first-come, first-serve basis. In contrast, 
only 3.9% indicated that they prioritize enrollment based on economic need. Since the 
programs are normally located in low-income communities, the site coordinators may not 


consider this issue as salient. 
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Student Recruitment and Retention 


After school programs have been shown to produce positive development and academic 
achievement outcomes for students who are consistently enrolled (Keating, Tomishima, 
Foster, & Alessandri, 2002; Hall, Yohalem, Tolman, & Wilson, 2003; Durlak et al., 2007). 
The recruitment and retention of certain targeted student populations, specifically those who 
are considered to be academically and economically disadvantaged, are thus central to the 
effectiveness of the program (Schwartz, 1996; Durlak et al., 2007). 


Recruitment of Students 


Site coordinators were asked to report on the different techniques that they use to 
recruit students. The most common techniques reported were teacher referral (89.1%), 
followed by the dissemination of flyers at the school or in the neighborhood (79.3%), and 
parent referral (75.1%). Many of the respondents also reported using student referrals 
(70.7%), and having the site staff (65.4%) and regular school day personnel (62.3%) do 
public relations for the site. The percentage of respondents who reported using the different 
techniques was consistent across urbanicity. In contrast, greater percentages of respondents 
who worked with middle school only sites than elementary only sites reported using each of 


the recruitment techniques. 


Site coordinators also reported on the populations that they target when recruiting 
students for their sites. The greatest percentage of respondents indicated that they target those 
students who are at-risk academically (77.3%). Moderately fewer respondents indicated that 
they target English Learners (59.5%) or students who are at-risk because of emotional and/or 
behavioral issues (53.1%). Little or no difference was present concerning the students 
targeted when examining the results by subgroup. 


While trying to achieve their targeted attendance goals, site coordinators must 
overcome a variety of obstacles. The most commonly reported obstacles to recruitment were 
student participation in other after school activities (25.4%), transportation (21.6%), lack of 
parental support (19.1%), and students supervising their siblings after school (17.2%). Other 
obstacles selected were student disinterest (16.8%), lack of staff (14.1%), cost (4.3%), and 
students work after school (2.8%). Concerning the subgroups, the ordering of the obstacles 
by percentage is similar to their ordering in general. Despite this, with the exception of cost, 
small to moderately greater percentages of respondents who worked with middle school only 
than elementary school only reported confronting each of the obstacles. 
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Retention of Students 


The results pertaining to obstacles for student retention were similar to those for student 
recruitment. While student involvement in other after school activities was still the most 
commonly selected obstacle (33.5%), the percentage of site coordinators who indicated that it 
was a retention issue increased by 8.1%. Furthermore, student disinterest appears to be more 
of an obstacle for retention than recruitment, with a small increase (5.3%) in the percentage 
of site coordinators who reported this issue (22.1%). When examining the responses by 
urbanicity, percentages were generally similar. Transportation is one exception with 
moderately more respondents from the suburbs indicating that it was a retention issue than 
those in the cities (28.2% and 19.2%, respectively). Furthermore, as with recruitment, 
respondents from the middle schools were more likely than were the respondents from the 
elementary schools to report that they confronted the different obstacles to retention. For 
example, moderately more middle school only respondents (31.7%) than elementary only 
respondents (12.9%) indicated that students supervising other siblings after school was an 


obstacle to student retention. 
Goals and Outcomes 


The specification of goals is a hallmark of quality after school programs (Chung, 2000; 
Latham & Yukl, 1975). Goals provide direction to programs (Patton, 1997), mediate 
performance, and regulate actions. Once established, administrators ought to design learning 
activities that contribute to the attainment of programmatic goals (Brophy & Alleman, 1991). 
In order to evaluate the impact of a program effectively, goals and activities should be 
aligned (Brophy & Alleman, 1991; Zais, 1976). The information garnered from the 
evaluation may then be used to facilitate decision-making in regards to funding, program 


design, and program improvement (Chung, 2000). 
Resources for Setting Goals and Determining Outcomes 


On the Part A questionnaire, the program directors reported on the stakeholders and 
data sources that they turn to when setting goals or determining outcomes. The greatest 
percentages of respondents indicated that they solicited feedback concerning goals and 
outcomes from program level staff (86.5% and 83.0%), day school administrators (86.2% 
and 80.8%), and/or site supervisors (84.3% and 81.1%). Likewise, three of the data sources 
were reportedly used by the vast majority of respondents. These included state achievement 
scores (84.9% and 85.9%), after school program attendance (81.7% and 82.1%), and site 
observations (72.1% and 71.5%). Few if any of the program directors indicated that they did 
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not seek feedback from any stakeholders or use any data sources when setting goals and/or 
determining outcomes. 


Goal Setting and Outcomes at the Grantee Level 


Both ASES & 21° CCLC guidelines require their grantees to have a strong academic 
focus (CDE, 2008). Therefore, it is not surprising that the majority of program directors 
responded on the Part A questionnaire (See Table 14) that their program goals for the 2009- 
10 school year included both academic improvement (93.6%) and improved homework 
completion (80.6%). The majority of respondents (86.9%) also reported that they set goals 
for improving program attendance during the 2009-10 school year. 


Table 14 
Goals Set and Progress Made during the 2009-10 School Year, Part A Questionnaire 


Progressed Set goal, but Failed to 

Goal Set goal Met goal towards goal _ not evaluated progress 
Academic improvement 93.6% 15.4% 49.1% 35.5% 0.0% 
(n = 298) 
Improved day school 68.0% 27.8% 36.9% 34.8% 0.5% 
attendance (n = 275) 
Improved homework 80.6% 29.3% 40.9% 29.3% 0.4% 
completion (n = 279) 
Positive behavior change 69.3% 19.8% 50.8% 28.3% 1.1% 
(n = 270) 
Improved program 86.9% 43.7% 35.3% 19.3% 1.7% 
attendance (n = 274) 
Increased skill 62.3% 23.5% 45.1% 30.2% 1.2% 


development (n = 260) 


For program directors who reported setting one or more of the above goals, they were 
asked whether these goals were met, progressed toward being met, were not yet evaluated, or 
failed to progress. Although academic improvement was the most commonly reported goal, it 
was also the goal that was most likely to not be evaluated (35.5%), and when evaluated, it 
was the goal least likely to be met (15.4%). This result may have to do with the timing of 
data collection for this study, which took place prior to the release of the state achievement 
score results for the 2009-10 school year. It is also interesting to note that none of the 
program directors who reported setting academic improvement as a goal reported that their 
programs failed to progress towards this goal. 
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Goal Setting and Outcomes at the Site Level 


Results for goal setting and progress are represented at the site-level in order to allow 
for examination of the data by urbanicity as well as grade span. The data originates from 
responses provided by the program directors to the Part A questionnaire. These responses are 
then linked to each of their sites that completed a Part B questionnaire (See Appendix A). 


Overall, academic improvement was the most commonly set goal (93.3%), followed by 
homework completion (81.1%), and positive behavior change (73.0%). Furthermore, 
moderate to large differences were present between the three types of location, with most of 
the goals more likely to be established for the suburban sites than for the city or town/rural 
sites. Prominent among these differences was increased skill development, with a much 
larger percentage of sites in the suburbs having this goal (77.8%) than in the cities or suburbs 
(50.9% and 44.8%, respectively). Small differences were also present based on grade span. 
Elementary school sites were slightly more likely than middle school sites to have academic 
goals set for them. Conversely, middle school sites were slightly more likely than elementary 


school sites to have behavioral goals set for them. 


Based on the program director reports, more than half of the sites that had goals set 
(57.5% or higher) also had them evaluated. Likewise, the majority of these sites (57.4% or 
higher) either met or were showing progress towards meeting their set goals. Interestingly, 
the sites located in the cities were more likely than those sites in the other locations to have 
their academic goals evaluated, while those in the town/rural areas were more likely than the 
sites in the other locations to have their behavioral goals evaluated. This trend was also true 
when looking at whether sites were reported to have met or progressed towards their goals. It 
is also important to note that, with the exception of homework completion and improved 
program attendance, most of the urbanicity differences were small. Small differences were 
also found by grade span, with the middle school sites more likely than the elementary school 
sites to have their program director report that their goals were being evaluated. This was 


also true concerning whether sites were meeting or progressing towards their goals. 
Programming and Activities 


Both the ASES and 21° CCLC programs require after school sites to provide academic 
assistance and enrichment to their participants. For example, programs must include tutoring 
and/or homework assistance, must support the school curriculum, and must provide 
additional activities that complement the regular school day program. This following presents 


the findings concerning programming emphasis from the Part B questionnaire. 
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Programming Emphasis 


The site coordinators were asked to rate the level of emphasis that they placed on 
different activities and outcomes using a four-item scale ranging from “Not at all” to “A 
great deal” (See Appendix A) As was previously noted, greater percentages of sites had goals 
set that concerned academic issues. Therefore, it is not surprising that high percentages of 
site coordinators reported placing a great deal of emphasis on homework assistance (91.8%) 
and academic achievement (86.5%). When examining the results by urbanicity, small to 
moderate differences were present for tutoring with more sites in the town/rural areas (60.7%) 
than the cities (50.1%) and suburbs (47.8%) placing a great deal of emphasis on this program 
feature. In addition, respondents in the town/rural areas were moderately less likely than their 
colleagues in the other locations were to place a great deal of emphasis on either program 
attendance or school attendance. The results by grade span had similar differences, with 
65.2% of respondents who worked with middle school only sites and 46.9% of those working 
with elementary only sites emphasizing tutoring a great deal. In contrast, the respondents for 
the elementary schools were slightly more likely than those at middle schools to place a great 
deal of emphasis on program attendance (86.5% and 76.9%) or school attendance (65.7% and 
59.4%). 


Alignment between the site coordinator responses about programming emphasis and 
the goals and outcomes reported by their program directors was examined (See Appendix A). 
These areas of emphasis involved academics, homework, program attendance, and school 
attendance. In each instance, almost all of the site coordinators who stated that they placed a 
great deal of emphasis on one of these areas had their program director report that their site 
had a corresponding goal, which they had met or progressed toward meeting. For example, 
95.2% of sites where a great deal of emphasis was made concerning school attendance met or 


progressed towards that goal. 
Academic Activities 


Site coordinators were asked about the core academic activities that were offered at 
their sites during the 2009-10 school year (See Appendix A). Overall, moderate to large 
differences were present, with more respondents reporting that they were offering math and 
language arts/literacy activities (81.4% and 78.9%, respectively) than science and history 
activities (65.3% and 42.8%, respectively). Differences across urbanicity were small to 
minimal. Furthermore, small to moderate differences were present concerning grade span. 


The largest of these differences involved science activities; Over two-thirds (69.3%) of the 
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elementary respondents reported that they offer these activities, while only 52.2% of middle 
school respondents reported the same. 


The site coordinators were also asked to report whether they offer general academic 
assistance activities (See Appendix A). The majority of this type of activity that were 
reported by greater than half of the respondents also directly met the ASES and 21° CCLC 
program requirements. These included homework assistance (94.9%), academic enrichment 
(89.7%), and tutoring (60.7%). Interestingly, the majority of site coordinators also reported 
offering nutrition education activities (71.6%). When examining the results by subgroup, 


little or no difference was found for many of the general academic activities. 
Non-Academic Enrichment Activities 


Despite the recent emphasis on developmental issues by researchers and policy 
makers, few of the developmental activities listed on the Part B questionnaire were offered at 
a majority of the sites during the 2009-10 school year (See Appendix A). Only slightly more 
than half of the site coordinators reported offering activities such as school safety (57.1%) 
and youth development (53.7%). Only small or minimal differences were found when 
examining the results across each of the urbanicity areas. In addition, site coordinators whose 
sites served middle school only were slightly more likely than those whose sites served 
elementary school only to report that they offer the developmental non-academic activities. 
One exception was school safety, which was reported moderately more by site coordinators 


who served elementary only (61.0%) than by those who served middle school only (44.9%). 


Site coordinators were also asked about whether they offered some other common 
non-academic activities (See Appendix A). High percentages responded that they offer 
physical activity/sports (92.1%), arts/music (89.5%), and recreational activities (87.4%). 
Furthermore, more than half of the site coordinators reported offering activities focused on 
computer/Internet skills (54.6%). With the exception of computer/Internet skills, differences 
concerning the subgroups were generally small. Respondents in the town/rural areas (65.3%) 
were moderately more likely than were those in the suburbs (56.1%) or cities (50.3%) to 
report offering this type of activity. Similarly, site coordinators who worked with middle 
school only (61.5%) were more likely than those who worked with elementary school only 
(52.4%) to report that they offer activities on computers/Internet skills. 


Staffing 


Literature reveals that a key characteristic of quality after school programs is having 
well qualified staff members that are sufficiently trained for their respective roles (Fashola, 
2002). In order to maintain the quality of an after school site, it is important that staff 
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turnover is low, which can be achieved by maintaining staff motivation and morale in 


providing staff with opportunities for training and learning. 
Staff Qualifications 


Program directors were asked about the required qualifications for their position, as 
well as for the program’s site coordinators and site staff (See Table 15). For program 
directors, the qualifications mentioned by the greatest percentage of respondents were prior 
administrative experience (71.2%), followed closely by general work with children (68.3%). 
With regard to site coordinators, the qualifications mentioned most frequently were general 
work with children (77.9%) and teaching experience (50.6%). Finally, the greatest 
percentage of respondents indicated that members of the site staff were required to be an 
instructional aide (73.4%) and/or a paraprofessional (66.7%). The result for site staff is not 
surprising considering that the ASES program requires staff members who directly supervise 
students to meet their districts’ requirements for instructional aides, while the 21° CCLC 


program requires site staff to meet federal requirements for paraprofessionals. 


Table 15 


Qualifications Required for After School Staff, Part A Questionnaire 


Program director Site coordinator Site staff 

Qualification (n = 312) (n= 312) (n= 312) 

Teaching certification 63.5% 27.2% 11.9% 
Prior administrative experience 71.2% 43.9% -- 
Teaching experience 59.3% 50.6% -- 
General work with children 68.3% 77.9% -- 

Paraprofessional -- -- 66.7% 

Instructional Aide -- -- 73.4% 


Site Management 


Site coordinators were asked about their style of management. Across all sites, 
respondents indicated that they were much more likely to employ a collaborative approach 
(81.0%) than a top-down or other type of management approach (12.5% and 6.5%, 
respectively). Differences across the urbanicity areas were generally small, with slightly 
greater percentages of site coordinators in the cities (83.0%) reporting that they use a 
collaborative approach than in the suburbs and town/rural areas (79.7% and 77.0%, 
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respectively). In contrast, there was no real difference concerning the collaborative approach 


when examining the results by grade span. 
Types of Staff Employed and their Retention 


Site coordinators reported on the types of staff who worked at their sites on a typical 
day during the 2009-10 school year. Almost all of the respondents (99.3%) reported that at 
least one site coordinator was present at their site on a typical day. Most sites also reported to 
have at least one credentialed staff member (60.6%) and/or at least 3 staff who were 
classified as paraprofessionals/instructional aides (87.6%) at their site on a typical day. 


When examining the results by urbanicity and grade span, some variations showed up. 
For example, site coordinators in the town/rural areas (70.8%) were moderately more likely 
than their colleagues in cities (54.9%) to report that they have at least one credentialed staff 
member on site. However, they (77.3%) were moderately less likely than their colleagues in 
the cities and suburbs (89.3% and 91.0%, respectively) to report that they had at least three 
paraprofessionals/instructional aides on site. When looking at the grade span results, it 
appeared that moderately more middle schools than elementary schools (73.7% and 56.1%, 
respectively) employed credentialed staff. 


Site coordinators were also asked to report on which types of staff they had to replace 
during the 2009-10 school year. Overall, the majority of respondents (74.4%) indicated that 
they had to replace at least one staff member who was classified as a paraprofessional or 
instructional aide. In contrast, less than one-third reported that their site had to replace a 
credentialed staff member or a site coordinator (22.6% and 30.0%, respectively). When 
examining the results by urbanicity, only small differences were present concerning the 
replacement of site coordinators or credentialed staff. However, sites located in the suburbs 
(80.8%) were moderately more likely than those in the cities or town/rural areas (73.6% and 
66.5%, respectively) to replace one or more staff members who were paraprofessionals or 


instructional aides. 


The results by grade span showed middle school respondents (36.2%) were slightly 
more likely than the elementary school respondents (28.2%) to report that their site had 


replaced a site coordinator during the current school year. 
Techniques used to Recruit Staff by Urbanicity and Grade Span 


Site coordinators were asked about the techniques they used for the recruitment of staff. 
According to the respondents, they would most likely offer support for educational goals 
(34.0%) and least likely use benefits (13.9%) as a technique for recruitment. This also holds 
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true when looking at the results by urbanicity, except concerning the use of opportunities for 
promotion; with moderately fewer site coordinators in the town/rural areas (17.8%) 
reportedly using this technique than their colleagues in the suburbs and cities (30.9% and 
32.7%, respectively). 


When examining the results for grade span, respondents from middle school sites were 
slightly more likely to use salary (34.0%) as a recruitment technique than were those from 
elementary sites (28.5%). In contrast, elementary school respondents were slightly more 
likely to use opportunities for promotion (30.3%) as a recruitment method than were middle 
school respondents (26.6%). 


Techniques used to Retain Staff by Urbanicity and Grade Span 


The majority of respondents (57.1%) indicated that they used recognition of staff for 
retention. Interestingly, the site coordinators from the town/rural areas (22.9%) were 
moderately less likely than their colleagues in the cities and suburbs (60.1% and 58.8%, 
respectively) to utilize opportunities for promotion as a technique, instead their second most 


popular strategy is support for educational goals (33.5%) 


Site coordinators in middle and elementary schools used similar techniques to retain 
staff, but the greatest difference involved the use of salary, with slightly more site 
coordinators in the middle schools (35.9%) utilizing this retention technique than those in the 


elementary schools (28.8%). 
Professional Development 


Professional development for after school staff has been repeatedly shown to positively 
affect the outcomes of youth either directly (Early et al., 2005, as cited in Harvard Family 
Research Project, 2005, 2006) or indirectly through improvements in staff competency and 
program quality (Harvard Family Research Project, 2005, 2006). In order to maximize the 
benefits of professional development opportunities, it is important that these trainings not 
only be aligned with program goals but they should also focus on individual and 
organizational development (Alexander 1986; Harvard Family Research Project, 2006; 
Partee, 2003). 


Staff Offered Professional Development 


Both program directors and site coordinators were asked whether any staff members, if 
any, were given opportunities to participate in professional development during the 2009-10 
school year. Whereas only 5.3% of program directors claimed that no professional 


development was offered, almost a quarter of site coordinators (23.8%) stated the same. 
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Based on results from both the program directors and site coordinators, the two types of staff 
who receive the most opportunities for professional development are site coordinators 
(96.5% and 93.4%, respectively) and site staff who are paraprofessionals/instructional aides 
(92.0%, respectively) In addition, over three-quarters of the program directors (78.3%) 
indicated that they were also given opportunities for professional development. When 
examining the results by urbanicity, the responses were very similar. One exception involved 
opportunities for volunteers, with site coordinators in the town/rural areas (11.1%) reporting 
moderately fewer opportunities than those in the cities and suburbs (21.1% and 22.4%, 
respectively). In addition, respondents whose sites served middle school students were 
moderately more likely to report offering professional development opportunities to their 
credentialed staff than respondents who served elementary students (41.1% and 32.7%, 


respectively). 
Organizations Providing Professional Development 


Program directors and site coordinators were also asked about the organizations that 
provided the professional development during the 2009-10 school year (See Table 16). The 
after school programs and school districts were the top two organizations chosen by both 
program directors (78.7% and 65.4%, respectively) and site coordinators (82.6% and 50.9%, 
respectively). The majority of program directors also reported that they utilize their Regional 
Lead Office and/or County Office of Education (61.5% and 55.6%, respectively) for 
professional development. In contrast, only about one third of site coordinators reported that 
they use these two resources (27.5% and 37.4%, respectively). The least reported resource by 
both groups of participants was a federal agency. 
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Table 16 


Organizations that Provided Professional Development during the 2009-10 School Year, Part A and Part B 
Questionnaires 


Grantee level Site level 
Organization (n = 286) (n = 971) 
The California Department of Education 21.0% 23.6% 
Federal agency 1.7% 2.0% 
California After School Resource Center (CASRC) 24.5% 18.0% 
Regional Lead Office 61.5% 27.5% 
County Office of Education 55.6% 37.4% 
School district 65.4% 50.9% 
Day school staff (e.g., teachers, administrators 40.6% 34.1% 
After school program (e.g., staff development dept., 78.7% 82.6% 
traveling leadership staff, site coordinator) 
Nonprofit organization 39.5% 33.8% 


Types of Professional Development 


Both the Part A and Part B questionnaires asked about the types of professional 
development offered during the 2009-10 school year. Similar percentages of program 
directors and site coordinators reported offering each of the different types of professional 
development. In both cases, most of the respondents reported offering trainings and/or 
workshops (91.3% and 93.0%, respectively), program-level meetings for leadership staff 
(90.2% and 91.8%, respectively), and site-level meetings for site staff (89.5% and 85.3%, 
respectively). Job preparation for leadership staff was the least mentioned, with only about 
half of the program directors and site coordinators (54.5% and 56.5%, respectively) 


mentioning its availability. 
Professional Development Topics 


Finally, both program directors and site coordinators were asked to report on the 
professional development topics offered during the 2009-10 school year (See Table 17). In 
general, the program directors and site coordinators were very similar in their responses. 
Most of these respondents reported that classroom management (89.5% and 89.7%, 
respectively), behavior management (88.8% and 87.4%, respectively), and federally 
mandated trainings (82.5% and 81.4%, respectively) were offered. The only topics where 
program director and site coordinator reports differed moderately involved conflict resolution 
(58.0% and 69.0%, respectively) and technology (48.3% and 36.7%, respectively) training. 


47 


Table 17 
Professional Development Topics Offered During the 2009-10 School Year, Part A and Part B Questionnaires 


Grantee level Site level 

Professional development topic (n = 286) (n = 971) 
Federally mandated training (e.g., CPR, first aid) 82.5% 81.4% 
Information about human resources 54.5% 52.1% 
Background information about the ASP 71.3% 70.3% 
Site management 66.4% 63.7% 
Classroom management 89.5% 89.7% 
Behavior management 88.8% 87.4% 
Student motivation and/or engagement 78.0% 76.7% 
Conflict resolution 58.0% 69.0% 
Lesson planning 72.7% 76.8% 
Content-specific training 65.0% 62.6% 
Curriculum specific training 63.6% 59.3% 
Technology 48.3% 36.7% 
Working with families and/or the community 44.8% 45.0% 


Community Partnerships 


The partnering of community members with after school programs is beneficial for 
program efficiency and effectiveness. Current studies indicated that community partnerships 
are beneficial for program development, program sustainability, and maintaining program 
quality (C. S. Mott Foundation Committee on After-School Research and Practice, 2005). 
This section of the report presents the findings from the Part B questionnaire concerning the 
individuals and organizations with whom the sites partner and the roles that they play. 


Community Involvement by Urbanicity and Grade Span 


Site coordinators were asked whether there was any community involvement in their 
program during the current school year. Overall, approximately three-quarters (74.5%) of the 
site coordinators reported that they did have some form of community involvement in their 
program. Differences were small when comparing the results by location, with those in the 
town/rural areas (71.6%) being slightly less likely than those in the cities and suburbs (74.4% 
and 76.5%, respectively) to report having community involvement. Similarly, there was no 


difference by grade span. 
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Individuals and Organizations that Partner with the Sites 


Site coordinators were asked to report about the different community members that 
play a role at their site. Of the options provided on the Part B questionnaire, the majority of 
respondents indicated that parents (80.8%), school or district staff (66.7%), and high school 
students (60.8%) had played a role. When examining the results for the different urbanicity 
areas, site coordinators from the cities (25.2% and 29.6%, respectively) were moderately less 
likely than their colleagues in the suburbs (35.0% and 43.1%, respectively) and town/rural 
areas (29.1% and 38.5%, respectively) to report that the involvement of employees from 
local businesses or agencies. In contrast, site coordinators from the town/rural areas (70.8% 
and 25.5%, respectively) were moderately less likely than those in the cities (85.0% and 
49.3%) and suburbs (78.8% and 42.3%) to report the involvement of parents and college 
students. When examining the results across grade span, the biggest difference had to do with 
members of local nonprofit organizations, with moderately more site coordinators at the 
middle schools than elementary schools reporting on these partnerships (49.5% and 38.4%, 
respectively). 


Site coordinators were also asked to report on the different Local Education Agencies 
(LEAs) that played a role at their sites. Similar to the results for community members, the 
majority of site coordinators responded that the public schools and/or their school district 
played a role in the operation of their site (61.5% and 57.1%, respectively). With regards to 
the involvement of the County Offices of Education, site coordinators in the town/rural areas 
(59.6%) were moderately more likely than those in the suburbs were (30.7%) and much more 
likely than those in the cities (18.3%) were to report having these partnerships. In contrast, 
site coordinators in the cities (18.3%) were more likely than those in the suburbs and 
town/rural areas (19.3% and 9.9%, respectively) were to report having a college play a role at 
their site. When examining the results by grade span, the biggest differences had to do with 
the involvement of colleges and County Offices of Education. In both cases, the site 
coordinators who worked with the middle schools (27.3% and 38.6%, respectively) were 
moderately more likely than those who worked with the elementary schools (17.3% and 
26.3%, respectively) to report on the involvement of these LEAs. 


Different Roles at the After School Sites 


Finally, those site coordinators who reported having community partners were asked 
about the roles that these different types of community members played (See Table 18). In 
general, site coordinators were more likely to report the involvement of the LEAs rather than 


the parents or other community members in the different roles listed. For example, site 
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coordinators were moderately more likely to report that the LEAs (41.2%) provided 
professional development than did other community members or parents (18.2% and 1.1%, 
respectively). In contrast, greater percentages of site coordinators reported that parents 
(41.9% and 26.3%, respectively) helped provide goods/supplies or raise funds than did the 
LEAs (33.0% and 19.2%, respectively) or other community members (29.6% and 20.6%). 


Table 18 


Roles that Community Members Play at the After School Sites, Part B Questionnaire 


Other community 


LEAs Parents members 

Role (n = 942) (n = 942) (n = 942) 

Oversee management of ASP 34.4% 5.7% 6.1% 
Participate in data collection for evaluation(s) 36.4% 21.1% 8.7% 
Raising funds 19.2% 26.3% 20.6% 
Setting or revising program goals 36.1% 19.1% 11.8% 
Implementing programs 35.0% 9.9% 14.8% 
Providing goods or supplies 33.0% 41.9% 29.6% 
Recruiting site staff 29.7% 10.3% 17.6% 
Involved in the hiring process of staff 27.8% 1.5% 6.6% 
Involved in the staff review process 26.8% 3.4% 5.4% 
Providing professional development 41.2% 1.1% 18.2% 

Summary 


During the 2009-10 school year, 396 grantees received ASES and/or 21° CCLC 
funding from the CDE. Although most of these grantees received funding from only one of 
these programs, small percentages received funding from both ASES and 21° CCLC or from 
one or more of these programs and the ASSETs program for high school students. The 
grantees and sites generally targeted students considered academically at-risk and/or English 
Learmers. Recruitment and retention obstacles cited most often were the following: students 
being involved in other after school activities, transportation, lack of parental support, and 
students supervising other siblings after school. Those sites located in the cities were 
somewhat less likely than those in the other areas to report that transportation was an 
obstacle. Furthermore, site coordinators who worked with middle schools were more likely 
than those at elementary schools to confront most of the obstacles to recruitment and 


retention. 
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Overall, the sites funded by the ASES and 21* CCLC programs tended to use a 
collaborative management style. Furthermore, over three-quarters of the program directors 
reported that they solicited feedback for goals and outcomes from program-level staff, site 
supervisors, and/or day school administrators. Similarly, program directors most often used 
state achievement scores, after school program attendance, and site observations as their data 
sources in establishing their goals and determining their outcomes. Program directors for a 
majority of the grantees stated that their programs had academically oriented goals. Increased 
program attendance and improved homework completion are other goals that were frequently 
mentioned. These results were true across urbanicity and grade span. The majority of 
program directors also indicated that they evaluated their goals and that they had met or 


made progress towards each. 


As for activities offered, math and language arts were offered by most sites. Science 
and history activities were offered to a lesser extent. The top academic assistance activities 
were homework assistance, academic enrichment, and nutrition education. The top three non- 
academic activities were physical fitness/sports, arts/music, and recreation. Furthermore, the 
top developmental activities, which were each offered by a majority of the respondents, were 


school safety and youth development. 


In terms of staff qualifications, having general work experience with children was most 
important for staff in leadership positions (i.e., program directors and site coordinators). In 
contrast, it was most important for site staff who directly supervised students to meet the 
ASES and 21° CCLC requirements of being paraprofessionals and/or instructional aides. 
With respect to daily operations, the majority of sites claimed that they had at least one site 
coordinator, some credentialed staff, and/or at least three paraprofessionals/instructional 
aides working on a typical day. Interestingly, the sites located in the town/rural areas were 
more likely than sites located in other areas to report that they employed some credentialed 
teachers. Furthermore, sites located at schools that served middle school students were more 


likely than sites that served elementary school students to employ credentialed teachers. 


As for staff replacement rates (turnover), sites were more likely to replace staff 
members who were paraprofessionals and/or instructional aides than staff who were 
credentialed teachers or site coordinators. With respect to recruitment techniques, over one 
quarter of the site coordinators reported that their program supports educational goals, 
provides salary incentives, recognizes staff accomplishments, and provides opportunities for 
promotion. Interestingly, moderately greater percentages of respondents reported that they 
use recognition of staff and/or opportunities for promotion as techniques to retain staff. 
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For professional development, there were discrepancies at the grantee and site levels. 
Program directors were more likely than site coordinators to report that there were 
opportunities for professional development. Furthermore, program directors were moderately 
more likely than site coordinators to report that credentialed site staff and volunteers were 


offered professional development. 


As for organizations that offered professional development to the sites, more than 
three-quarters of the grantees and site coordinators stated that the after school programs were 
the most likely to provide such services. The top three types of professional development 
reported by the grantees and site coordinators were trainings/workshops, program-level 
meetings, and meetings at the after school sites. Furthermore, the top three topics were 
classroom management, behavior management, and federally mandated topics such as CPR 


and first aid. 


The great majority of sites funded by the ASES and/or 21° CCLC program maintained 
community partnerships. The most frequently reported individuals were parents, school or 
district staff, and high school students. Likewise, the majority of sites reported that they 
maintained community partnerships with the Local Education Agencies (LEAs) of public 
schools and school districts. LEAs were more strongly involved in more aspects of the ASES 
and 21° CCLC sites than were the parents and other community members. This included 
participating in many operational aspects at the sites including, but not limited to data 
collection, professional development, and site management. Parents and other community 


members were generally more involved in providing goods and raising funds for the sites. 


These findings will support the ongoing development of the profiling system and 
provide the necessary information for the grantee database to be monitored and merged with 


the data from the previous and upcoming coming years (years 1 and 3). 
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CHAPTER VI: 
DESCRIPTIVE RESULTS OF PROGRAM SITES—SAMPLE IV 


Results from Sample III provide general grantee characteristics. In order to conduct in- 
depth analyses to address the six evaluation questions through site visits, telephone 
interviews, and focus groups, a subsample of 40 ASES and/or 21 CCLC after school sites 
(Sample IV) was established. Respondents to this study sample include program directors, 
school principals, site coordinators, site staff, parents, and student participants. This chapter 
presents the results of the preliminary analysis of Sample IV in Year 2. According to the 
theoretical model for the study and similar to Sample III, the discussion of the Sample IV 
results will be framed under the categories of (1) program orientation, (2) program 
environment, and (3) instructional features. First, descriptive characteristics of the 


respondents are provided. 
Sample IV Respondent Characteristics 
Student Characteristics 


Among the students surveyed, 79.4 % were from elementary schools (n = 456), and 
20.6 % were from middle schools (n = 118). The majority of these students claimed to attend 
school regularly (92.9% for elementary students and 99.2% for middle school) and the after 
school program 5 days a week (87.8% for elementary school participants and 86.0% for 
middle school participants). Based on self-report, most elementary students attended the 
same school (86.3%) and the same after school program (67.7%) they were in during the 
prior year, while a little more than half (56.8%) of middle school students did so as well. 
Tables 19 and 20 display the student’s self-reported data on their attendance and grades, 
respectively. 


Table 19 
Self-Report Data for School and After School Program Attendance During the Prior School Year 


Elementary % Middle % 

Attendance history (n = 447) (n= 112) 
Attended the same school 86.3% 56.8% 
Attended the same after school program 67.7% 57.6% 
Attended another after school program 13.2% 27.1% 
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Table 20 
Self-Report Data for Grades Received 


Elementary % Middle % 
Reported grades (n = 447) (n= 112) 
Mostly A’s 36.2 49.1 
Mostly B’s 43.6 30.4 
Mostly C’s 12.5 17.9 
Other grades 7.6 2.7 


Over three quarters of the middle school and elementary student respondents reported 
to have earned mostly A’s and B’s. Having been recruited from low-performing schools and 
high-crime locations, this student population appeared to be performing higher than 


expected. 


On a multiple-choice, multiple-answer question, the surveyed students provided the 
reasons that they attend their after school programs. Over half (52.9%) of elementary 
participants chose, “My parents want me to go,” as one of their reasons. The second most 
common response for them was, “I get help with my homework,” (44.3% for elementary). 
These two reasons were also the most common responses for the middle school students, as 
48.3% of the surveyed middle school students selected these two options. Table 21 displays 


the options and students’ responses in percentage. 
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Table 21 


Reasons for Attending the After School Program 


Elementary % Middle % 


Reason (n = 456) (n= 118) 
My parents want me to go. 52.9 48.3 
The school suggested that I go. 9.2 11.0 
My friends go here. 27.9 26.3 
I do not want to go home. 12.9 17.8 
There are interesting things to do here. 21.9 40.7 
I get help with my homework. 44.3 48.3 
It is a safe place to be after school. 27.6 21.2 
I attended this after school program last year. 19.1 27.1 
I get to participate in physical activities, such as sports. NA 33.1 
Other 21.5 11.0 


Parent Characteristics 


Nine hundred and one parents participated in the survey. Most of them were parents of 
2™ to 6" graders. About 90% of them had children who had attended their programs for more 
than 6 months, and about 60% of the parents surveyed had children who had attended their 
program for more than a year. Nearly all K-9 parents (94.8%) reported that their children 
attended the program 5 days a week. 


Twenty-one percent of the parents (n = 880) picked up their children early from the 
program. Of those parents, 40.8% picked their children up early five days a week, 25.3% of 
them did so one day a week, and 19.0% of them did so two days a week. It should be noted 
that students who were picked up early might not have reaped the full benefits from the 
program because they left earlier than intended. 


Staff Qualifications 
The following presents qualifications of the staff members. 


Site coordinators. When asked to report their highest level of education, 35.3% of site 
coordinators reported that their highest level of education was “some college.” The next most 
common responses were that they held either an Associate’s degree (23.5%) or a Bachelor’s 
degree (also 23.5%). About one fifth (22.2%) of the site coordinators had teaching 
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credentials. And a small number (5.6%) of site coordinators were enrolled in a teaching 
credential program at the time they completed the survey. 


All of the site coordinators who were surveyed spoke English, while 72.2% also spoke 
Spanish. Nearly three quarters (72.2%) of the site coordinators had been working at their 
program for more than three years, while 22.2% had been there for one to three years. In 
addition, 52.9% reported that they had been involved in management at their current program 
for three years or more, and 41.2% were involved in management at their program for one to 
three years. When site coordinators were asked how long they have been working in 
community organizations in general, 76.5% reported that they worked in community 


organizations for more than three years. 


Site staff. Almost half (48.6%) of the site staff who were surveyed had completed some 
college education; 21.9% had a Bachelor’s degree, and 13.3% had an Associate’s degree. 
Most of the site staff (90.7%) indicated that they did not have teaching credentials, and most 


were not enrolled in a teaching credential program. 


Most of the site staff (94.1%) spoke English, while 58.8% spoke Spanish. Less than 
half of the site staff (40.3%) indicated that they had been working at their programs for one 
to three years, and a quarter of the site staff (25.2%) had been working there for over three 
years. When site staff were asked how long they have been working in community 
organizations in general, 54.6% reported that they worked in community organizations for 
more than three years; another 31.5% indicated that they worked in community organizations 


for one to three years. 


These findings suggest that, in general, the site coordinators and site staff have 
sufficient knowledge and experience to carry out their tasks. Since a majority of the students 


were Hispanics, it is helpful to have a good number of the site staff who speak Spanish. 
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Program Orientation 


In order to confirm the representativeness of the Sample IV programs, site coordinators 
and staff were again inquired about their goals and program orientation. To further expand 
Sample III findings, they were also asked to express their opinions on the alignment of 


program goals to program activities. 
Goal-Oriented Programs 


On a 4-point Likert scale ranging from 1 (strongly disagree) to 4 (strongly agree), the 
site coordinators were asked about the goals of their program. The majority of them strongly 
agreed that their program has clear goals for students and/or a clear mission statement for 
program’s vision. They also believed that the program activities had aligned to the goals, 
while slightly more than half (55.6%) strongly agreed that most staff members are aware of 
the program’s goals. Table 22 displays the means and distributions of the site coordinators’ 


answers regarding program goals. 


Table 22 


Site Coordinator Responses Regarding Program Goals 


Site coordinator 


(n = 18) 
% Strongly 
Survey prompt Mean agree 
Our program has clear goals for students. 3.67 66.7 
We have a clear mission statement that explains our program’s vision. 3.72 72.2 
We align our program activities to our goals. 3.50 55.6 
Most staff members are aware of our program’s goals. 3.50 55.6 
Most staff members adhere to our program’s goals. 3.44 44.4 


Innovative Management 


At the same time, it is also important to have program leadership that can motivate 
staff, provide positive organizational climates that validate staff commitment to program 
goals, and open the communication channels between after school, day school, parents, and 
the community (American Youth Policy Forum, 2006; Wright, Deich, & Szekely, 2006). 


On the same 1-4 Likert scale, most site coordinators strongly agreed that staff members 
met with each other regularly to discuss student development and to share techniques on 
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engaging the students. Most staff members agreed with the site coordinators—though there 
was a lower mean and a lower percentage for the site staff who responded “strongly agree,” 
especially regarding the statement about having regular staff meetings to help struggling 
students. Table 23 displays the comparisons between the site staff and site coordinators in 
their responses to these questions. In general, these data support Sample III findings that 
most sites incorporate a collaborative approach to management. 


Table 23 


Site Coordinator and Site Staff Responses Regarding Program Implementation 


Site coordinator Staff 
n=18 n=119 
% Strongly % Strongly 

Survey prompt Mean agree Mean agree 
Staff members meet with each other to share 3.72 722 3.08 26.3 
techniques on how to keep students engaged. 
Staff members meet with each other on a regular 3.47 52.9 2.95 24.4 
basis to discuss strategies to help students who are 
struggling. 


Local Partnerships 


In addition to advocating goal-oriented programs and innovative management, research 
on after school programs consistently associates family and community involvement with 
program quality (Owens & Vallercamp, 2003; Tolman, Pittman, Yohalem, Thomases, & 
Trammel., 2002,Wright, 2005). Through local partnerships, students can gain a sense of 
belonging in their communities, learn about different trades and careers, and obtain in-service 
training. They may also be encouraged to participate in community service projects, which 
encourage a sense of empowerment and pride in their respective communities and develop 
good citizenship. The next paragraphs will discuss how the after school programs in Sample 
IV have partnered with the host schools, the local communities, and the parents. 


Host school. When responding to questions regarding their programs’ partnerships 
with the day schools, more than half of the site coordinators strongly agreed that they were 
informed by the school administrators of important decisions and issues related to their 
program. Most of them also agreed that day school teachers were collaborative with their 
staff (M =3.06), and their staff was responsive to day school staff’s ideas and suggestions (M 
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= 3.39). Table 24 shows site coordinators’ answers to questions regarding the programs 
partnered with the day schools. 


Table 24 


Site Coordinator Responses Regarding Partnerships with the Day School 


Site coordinator (n = 18) 


Survey prompt Mean % Strongly agree 
Day school teachers are willing to collaborate with our staff. 3.06 27.8 
Our staff is responsive to ideas and suggestions from day school staff. 3.39 38.9 
School administrator(s) keep me informed of important decisions and 3.44 50.0 


issues that affect our program. 


Community. Site coordinators were asked seven questions regarding their program’s 
partnerships with the local communities. As shown in Table 25, the majority of site 
coordinators agreed that their programs had an established a system to connect with local 
communities, invite community members to participate in program events and meetings, 
encourage local education agencies to give input regarding after school activities, partner 
with the local community to publicize the programs, and have community members 
participate in special events. On the other hand, the majority of site coordinators did not 
agree that the local communities had supplied resources for the program or that community 
members had participated in curricular decision-making. 
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Table 25 


Site Coordinator Responses Regarding Partnerships with Local Communities 


Site coordinator (n = 18) 


% % 
Strongly strongly 

Survey prompt Mean agree % agree %disagree disagree 
Our organization has an established system 2.94 16.7 66.7 11.1 5.6 
to connect with the local community. 
Community members are invited to events 2.83 11.1 66.7 16.7 5.6 
and meetings at our program. 
Local education agencies (districts, schools, 2.71 0 66.7 27.8 0 
etc.) are encouraged to give input regarding 
after school activities. 
The program partners with the local 2.61 5.6 55.6 33.3 5.6 
community to publicize the program. 
Community members participate in 2.28 5.6 22.2 66.7 5.6 
curricular decision-making (consultation, 
evaluation, etc.). 
The local community supplies resources for 2.06 0 22.2 61.1 16.7 
after school activities. 
Community members participate in special 2.71 11.8 58.8 17.6 11.8 


events. 


Parents. Survey instruments examined the different perspectives of the site 
coordinators, staff, and parents, asking them about their opinions on the programs’ 
relationships with the parents. 


Parent participation. Of those 878 parents who responded, 59.6% of the parents had 
visited their children’s after school programs and 35.2% reported that they had participated in 


the program. 


Survey responses from the site coordinators, staff, and parents were mostly in 
agreement in terms of how parents participated in the program. Among the 303 parents who 
reported to have participated, a majority of them attended program events, many attended 
meetings hosted by the programs, and some volunteered in program activities such as field 


trips and back-to-school nights. 


The staff shared similar opinions. During the staff focus groups, staff members often 
provided examples of how they involved parents with their children’s program. One of the 
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staff said, “I think that’s why we have family night, you know, so we can get to know the 
parents a little bit better, for them to see what the parent—to see what the kids do because 
we’ ll have—the kid will show either their art or the work that they’ve done over the past 
couple of months. So, we try to involve the parents with the activities that the kids like to do 
here. Sometimes we’ll have them do the same activities to see exactly what it is that the kids 
do here.” 


There were also disagreements among the stakeholders. For example, while only 18.2% 
of parents strongly agreed that they provided feedback on program activities and curriculum, 
the majority (77.8%) of the site coordinators strongly agreed that the parents did, while more 
than half (52.1%) of staff strongly agreed. Table 26 displays their responses. 


Table 26 


Parent Participation in the Program, as Reported by Parents, Staff, and Site Coordinators 


Site 
Parents Staff coordinator 
n= 303 n=119 n=18 


% strongly %strongly % strongly 


Survey prompt agreed agreed agreed 
Attend meetings hosted by the program (parent-staff meetings, 60.4 56.3 72.2 
conferences, etc.) 
Attend program events (open houses, performances, etc.) 79.2 69.7 83.3 
Volunteer in program activities (field trips, etc.) 22.8 38.7 55.6 
Give feedback on program activities and curriculum 18.2 52.1 77.8 
Other 3.0 10.9 16.7 


Obstacles to participation. When the parents (n = 901) were inquired about the 
obstacles that had prevented them from participating in their children’s program, the most 
common answer (40.6%) was the time conflicts between the parents’ work and the program 
hours. Having to care for other children who were not in the program was the next common 
reason (19.9%) that made it difficult for the parents’ to participate. Additionally, 13.4% of 
parents listed that a language barrier has prevented them from taking part in their children’s 
after school programs. Table 27 shows the list of answers from the parents’ survey. 


61 


Table 27 


Barriers to Program Participation (n = 901) 


Barriers to parent participation % 
Language barrier 13.4 
Program location 1.6 
Lack of transportation Dis 
Staff make me feel unwelcome 1.6 
Program is held during my work hours 40.6 
Staff discourage me from participating 2.7 
I must take care of my children who are not in the program 19.9 
Other 8.1 


Program policies for parental involvement. Parents, site coordinators, and site staff 
were asked to respond to a set of items about the parents’ roles if and when they get involved 
in the program. Table 28 presents the results. 


At the policy level, site coordinators in general agreed that parental involvement was an 
integral component of the program (M = 3.17). However, they responded below 3 (based on 
the Likert score of 3 as Agree) on the item which states that the program has a defined plan 
for parental involvement. They scored even lower on parental input considered in decisions 
about programming (M = 2.89). 


Meanwhile, the parents, site coordinators, and staff generally agreed that the staff kept 
the parents informed with what was going on in the program (M = 3.39, 3.39, 3.09, 
respectively). They also agreed that the staff communicated with parents about children’s 
well-being (3.39, 3.5, 3.22). However, though the parents and site coordinators agreed that 
staff members communicated with parents about how a parent can help their child learn 
(3.14, 3.44), the site staff barely agreed (2.92). 


It is interesting to note that the results from the parent surveys indicated that the parents 
held a more positive view of their involvement with the programs than the staff and the site 
coordinators did. It is also worthwhile to note a few discrepancies among the different 
stakeholders. While the majority of the site coordinators reported that their staff met with the 
parents regularly, 33.6% of staff disagreed they did so. In the same line, most of the parents 
believed their input was considered in decisions about after school programs, but 22.2% of 
site coordinators disagreed. 
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Table 28 


Parent Involvement in the After School Program 


Parents Site coordinator Site staff 
n = 879 n=18 n= 119 


% Strongly % Strongly % Strongly 
Survey prompt Mean agree Mean agree Mean agree 


Parent involvement is an integral - - 3.17 27.8 - - 
component of the program. 


Our program has a clearly - - 2.94 22.2 - - 
defined plan for parent 
involvement. 


The staff meets with the parents - 3.11 27.8 2.63 15.1 
on a regular basis 


Parents’ input is considered in 3.21 34.9 2.89 11.1 - - 
decisions about after school 
programming. 


Staff members keep parents 3.39 50.3 3.39 44.4 3.09 27.7 
informed about what is going on 
in the program. 


Staff members communicate with 3.39 51.8 3.50 55.6 3.22 35.3 
parents about children’s well- 
being. 


Staff members communicate with 3.14 37.9 3.44 44.4 2.92 19.5 
parents about how a parent can 
help their children learn. 


Staff members communicate with 3.38 48.7 - - - - 
parents about their children’s 
behavior at the program. 


Parents are encouraged to give 3.10 33.2 - - - - 
input on rules for appropriate 
behavior at the program. 


Parents are kept informed about 3.25 42.3 - - - - 
how their child is doing in the 

academic activities at the after 

school program. 


Next, program environment at these sites were examined. 
Program Environment 


The program environment focuses on how the structure of the after school program 
creates an atmosphere conducive to positive academic achievement and self-esteem for 


students. 
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Program Climate 


The most important feature of the program environment is safety and security within 
the indoor and outdoor space (National Institute on Out-of-School Time, 2002; New Jersey 
School-Age Care Coalition, 2002; Philadelphia Youth Network, 2003; St. Clair, 2004; 
Wright, Deich, & Szekely., 2006). The main aim is to make sure that students are in a safe, 
supervised environment that provides ample resources for physical and emotional growth. 
Students, parents, site coordinators, and site staff were asked about the climate at their after 
school program. The questions touched upon the topics of physical and psychological safety, 
relationships amongst the staff and students, and students’ feelings of belonging. 


Survey findings revealed that students in after school programs were in fact provided 
with a safe, supervised environment. When asked whether they agreed with the statement, “I 
feel safe in the after school program,” the majority of students strongly agreed (49.7%) that 
they felt safe (M = 3.27, n = 565). 


The establishment of this physically safe environment helps the development of 
positive relationships. 


Relationships 


Quality afterschool settings can provide the opportunities for students to develop 
relationships that signify positive, influential connections with adults and with peers 
(Beckett, Hawken, & Jacknowitz, 2001; Birmingham, Pechman, Russell, & Mielke, 2005; 
Huang, 2001). 


Staff-student. Aside from primary-based interactions within the home, the interaction 
between the staff members and students is vital for demonstrating affirmative adult-student 
relationships (Birmingham et al., 2005; Bodilly & Beckett, 2005; Harvard Family Research 
Project, 2004; New Jersey School-Age Care Coalition, 2002). When staff members are able 
to form personable, one-on-one relationships with students through daily conversations and 
engagement (St. Clair, 2004), this initiates a sense of community and belonging for the 
students because they are personally bonded to staff members (Wright et al., 2006). 


Survey results indicate that students perceived that they had a positive relationship with 
after school staff. They stated that they felt the site staff cared about them (M = 3.12, n = 
561) and listened to what they had to say (M = 3.13, n = 556). Parents had a similar view of 
the relationship between their children and adults in the program. They felt that students had 
positive relationships with both the site staff (M = 3.31, n = 870) and volunteers (M = 3.25, 
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n= 858). It is of interest to note that parents also felt that their children had a positive 
relationship with the day school staff (M = 3.35, n = 871). 


Student-Student. Positive peer relationships and friendships are key ingredients in 
shaping students’ social-emotional development (Halpern, 2004; Harvard Family Research 
Project, 2004; Yohalem, Wilson-Ahlstrom, & Yu, 2005). Students need to interact with each 
other, building strong “partnerships” based on trust and respect with their peers (Yohalem, 
Pittman, & Wilson-Ahlstrom, 2004). Healthy interaction with other students of various ages, 
and being involved in age appropriate activities, helps students to demonstrate appropriate 
problem-solving strategies, especially during times of conflict (Wright et al., 2006). The 


following excerpt from a student focus group helps express the students’ point of view: 


Interviewer: Do you think the program has helped you get along better with your 
peers? 

Student 1: They help us with our conflict with them. 

Student 2: They help us get along better by working as a team. 

Student 3: We can participate in class more. 

Student 4: And make new friends. 

Interviewer: Make new friends, uh-hm. 

Student 4: Even when you're shy. 

Interviewer: Uh-hm. So Student 1 mentioned that they help them with conflicts. 


[Do you have] any examples? 

Student 1: Like if, let’s say you’re in an argument inside class with someone and 
the Mentor will (pause) will like tell you guys to step outside. And 
then he or she will help you guys talk it out and like will help (pause) 
will give advice or solutions to help us stop the conflict. 


As with their relationship with adults at both the program and the day school, parents 
reported that students had positive relationships with their peers in the after school program 
(M = 3.36 n=873) In addition, they reported that their children also had positive 
relationships with their peers in the day school (M = 3.38, n = 871). 


Opportunities to Belong. Students reported fairly strong feelings of belongingness 
with respect to the program. They felt close to the staff (M = 2.99, n = 565) and were happy 
to attend the program (M = 3.09, n = 565). The following are some of their expressions: 


e “If you got hurt in the classroom, they [staff] tell you if you want to go to room 3 or 
sometimes they have band-aids...They keep you safe.” 


e “TI come to this program because I like the game, I like the people who keep us safe, 
and my friends come here.” 
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e “T like it because if I don’t attend, who’s gonna take care of me? I have nowhere 
else...” 


Next, resources at the program will be examined. 
Resources 


In order to promote the physical and emotional wellness of students, adequate materials 
and comfortable space is needed for staff members to conduct a range of stimulating 
activities. The indoor and outdoor space should be used appropriately, catering to the activity 
being carried out (e.g., sports, creative arts, and eating), so that the goals of the activities can 
be sufficiently met (New Jersey School-Age Care Coalition, 2002; Philadelphia Youth 
Network, 2003). 


Staff members and site coordinators were asked several questions regarding the 
resources available at their after school programs. Generally, both stakeholders agreed that 
they had enough materials such as books, papers, and pens and that they had enough time to 
work with students. They also seemed to agree that they had enough space to conduct their 
work. In general, the site coordinators appeared to have a more positive view regarding 
resources available to them. For example, site coordinators were more likely to report that 
they had enough staff/personnel and enough resources than the program staff were. See 
Table 29 for means and distributions for both stakeholders. 


Table 29 


Resources available at the program reported by Site Coordinator and Site Staff 


Site coordinator Site staff 
(n=17) (n= 119) 
% Strongly % Strongly 
Survey prompt Mean agree Mean agree 

Our site has enough materials, such as books, papers, 3.61 72.2 3.15 36.8 
pens, etc. 
Staff members have enough time to work with 3.28 38.9 3.10 32.8 
students. 
Staff members have enough time to prepare for 2.89 16.7 2.89 27.0 
activities. 
Our site has enough staff/personnel. 3.28 44.4 3.08 28.0 
Our site has enough space for students to do their 3.17 44.4 3.03 27.7 
work. 
Overall, we have enough resources. 3.50 50.0 2.92 23.1 
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Instructional Features 


When programs employ a variety of research-proven teaching and learning strategies, 
they can help staff members increase engagement among students with different learning 
styles (Birmingham et al., 2005). Contrarily, a failure to design activities that meet the needs 
and interests of students may result in reduced program attendance (Sepannen et al., 1993). 
In order to induce the intended impact of a program effectively and make sure that students 
receive the intended enrichment, interesting and engaging activities have to be available for 
students to participate. 


Students’ level of engagement in activities. Students were asked to respond to how 
often they participated in certain activities; they could respond on a four-point Likert scale 
ranging from 1 (never) to 4 (every day). Survey results showed that “homework,” “sports and 
games,” and “math” have the highest percentage of students stating that they carried out 
those activities every day. This supports the Sample III findings that high percentages of site 
coordinators reported placing a great deal of emphasis on homework assistance (91.8%), 
academic achievement (86.5%), and physical activity/sports (92.1%). Table 30 presents 
students’ reports on their levels of engagement in activities. 


Table 30 


Students’ Levels of Engagement in Activities 


Activity Mean % Everyday 
Homework (n = 563) 3.45 65.7 
Reading (n = 561) 2.78 34.4 
Writing (n = 559) 2.75 33.5 
Math (n = 561) 3.01 47.8 
Science (n = 550) 2.18 18.7 
Arts (dance, drawing, etc.) (n = 554) 2.70 34.7 
Sports and games (n = 553) 3.26 55.9 
Use the computer (n = 558) 2.12 20.1 


Staff Efficacy 


Literature reveals that a key characteristic of quality after school programs is having 
well-qualified staff that are sufficiently trained for their respective roles so that they are 
confident in engaging their students in activities (Fashola, 2002). Staff and site coordinators’ 
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responses on the surveys, interviews, and focus groups allowed the study team to investigate 
staff efficacy in more detail. First, the roles that the staff played in the after school program 


are explored. 


Roles played by staff in the after school program. When site staff were asked what 
role they played in the after school program, the majority of them reported that they spent 
time helping students with their homework and tutoring students. It is also important to note 
that the staff also reported to engage with students in other enrichment activities such as 


recreational activities and physical education. Findings are presented in Table 31. 


Table 31 
Roles Played by Staff in the After school Program 


Site staff 
(n =119) 
Role % 

Homework help 82.4 
Tutoring 52.1 
Language arts instruction 26.9 
Math instruction 42.9 
Science instruction 18.5 
Visual & performing arts instruction 28.6 
Physical education 49.6 
Other recreational activities 50.4 
Parent/community outreach 26.9 
Other 18.5 


Support for Efficacy 


As stated in Program Orientation, staff perceived that they received sufficient support 
in terms of materials, space, and time to work with students, even though there is an 


indication that more preparation time is needed. 


Previous research in program quality has posited that it is essential for staff to have 
qualifications such as training and higher levels of education in order to implement best 
practices at school (Wiss, 2006). These staff appeared to be well qualified for their roles as 
the majority of them reported to have an Associate, Bachelor, or Master degree. Furthermore, 


staff efficacy is enhanced when the site staff meet together to share techniques on engaging 
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students and to collaborate to resolve student academic and behavioral challenges. Regarding 
program implementation and support, as shown in Table 32, staff also appeared to agree that 
they met with each other to share techniques and strategies to help students to be engaged 
and help those students who were struggling 


Table 32 


Program Implementation and Support 


Site staff 
% 
Strongly 

Survey prompt n Mean agree 
Staff members meet with each other to 119 3.08 26.3 
share techniques on how to keep 
students engaged. 
Staff members meet with each other on a 119 2.95 24.4 


regular basis to discuss strategies to help 
students who are struggling. 


Professional Development 


Professional development further enhances staff efficacy. Staff generally reported that 
they received training annually, that the training offered was aligned with program goals, and 


that they felt the training enhanced their competencies. 


In general, site coordinators appeared to have a more positive view of the professional 
development offered compared to the staff’s view of the training they received. For example, 
a higher percentage of site coordinators strongly agreed that staff members were offered 
professional developmental annually than the site staff. A higher percentage of site 
coordinators also strongly agreed that professional development sessions helped the site staff 
to do their job better than the staff members. It is also interesting to point out that a higher 
percentage of site coordinators strongly agreed that they may request additional training for 
career advancement than were the site staff. Table 33 displays the means and distributions of 
both stakeholders. 
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Table 33 


Professional Development Responses from both Site Coordinators and Site Staff 


Site coordinator Site staff 
(n=17) (n = 119) 
% Strongly % Strongly 
Survey prompt Mean agree Mean agree 
Staff members are offered opportunities for 3.59 58.8 3.09 25.0 
professional development annually. 
Professional development offered is aligned with 3.47 52.9 3.15 26.3 
program goals. 
Staff members receive training prior to teaching 3.24 41.2 3.04 28.0 
content-specific activities. 
Staff members are offered training in behavior 3.35 52.9 3.00 19.7 
management. 
Staff members may request additional training 3.12 35.3 2.94 20.7 
for career advancement within the organization. 
Professional development sessions help site staff 3.53 58.8 3.11 26.3 


to do their jobs better. 


Although the site staff generally seem to be satisfied with the professional development 
received, they were likely to report that they would like to receive additional training on 
behavior management, using age appropriate teaching techniques, teaching academic skills, 


and explaining ideas to improve student understanding. See Table 34 for details. 
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Table 34 
Areas of Training that Site Staff Would Like to Receive 


Site staff 
(n = 119) 
Area of training % 

Teaching academic skills (language arts, math, science, etc.) 47.1 
Explaining homework materials 22.7 
Explaining ideas to help improve student understanding 46.2 
Using age-appropriate teaching techniques 47.9 
Implementing non-academic activities (visual arts, performing arts, sports, etc.) 31.9 
Managing behavior in the classroom 48.7 
Building leaderships skills (for site staff) 31.9 
Communicating with my students’ parents 26.1 
Communicating with my students’ day school teacher(s) 26.9 
Communicating with school administrators at my site 20.2 
Other 3.4 


Overall, the respondents indicated that these after school programs offered support in 
enhancing staff efficacy. 


Fostering Positive Youth Development 


Recently, support for the Positive Youth Development Approach has continued to build. 
This approach suggests that helping young people achieve their full potential is the best way 
to prevent them from engaging in risky behaviors (Larson, 2006). Positive youth 
development is both a philosophy and an approach to policies and programs that serve young 
people, focusing on the development of assets and competencies in all youth. After school 
programs that promote positive youth development give youth the opportunity to exercise 
leadership, build skills, and get involved (Larson, 2000). Karen Pittman (2008), Executive 
Director of the Forum for Youth Investment, identified the following key features as essential 
for the healthy development of young people: physical and psychological safety, appropriate 
structure, supportive relationships, opportunities to belong, positive social norms, support of 
efficacy and mattering, opportunity for skill building, and integration of family, school, and 
community efforts. This study’s preliminary findings on these key features at the Sample IV 


sites are presented in this section. 
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Table 35 provides descriptors of the eight features of positive developmental settings 
(National Research Council and Institute of Medicine, 2002). This section of the report will 
examine whether and/or how the after school program sites under study have adhered to 
these setting features. In addition, the report will also briefly discuss site staff’s perception of 
high expectations, the importance of future aspirations, and the fostering of students’ 


academic self-efficacy, cognitive competence, and socio-emotional competence. 


Table 35 


Features of Positive Development Settings 


Feature Descriptors 


Physical and psychological safety Safe and health-promoting facilities that increase safe peer 
group interaction and decrease unsafe or confrontational 
peer group interaction. 


Appropriate structure Limit-setting, clear and consistent rules and expectations, 
firm-enough control, continuity and predictability, clear 
boundaries, and age-appropriate monitoring. 


Supportive relationships Warmth, closeness, connectedness, good communication, 
caring, support, guidance, secure attachment, and 
responsiveness. 

Opportunities to belong Opportunities for meaningful inclusion, regardless of one’s 


gender, ethnicity, sexual orientation, or disabilities; social 
inclusion, social engagement, and integration; opportunities 
for socio-cultural identity formation; and support for 
cultural and bicultural competence. 


Positive social norms Rules of behavior, expectations, injunctions, ways of doing 
things, values and morals, and obligations for service. 


Support for efficacy and mattering Youth-based, empowerment practices that support 
autonomy make a real difference in one’s community and 
are being taken seriously; practices that include enabling, 
granting responsibility, and meaningful challenge; and 
practices that focus on improvement rather than on relative 
current performance levels. 


Opportunities for skill building Opportunities to learn physical, intellectual, psychological, 
emotional, and social skills; exposure to intentional learning 
experiences; opportunities to learn cultural literacies, media 
literacy, communication skills, and good habits of mind; 
preparation for adult employment; and opportunities to 
develop social and cultural capital. 


Integration of Family, school, and community Concordance, coordination, and synergy among family 
efforts school, and community. 
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Safety, Caring Relationships, and Belongingness 


A feeling of physical and psychological safety is necessary for intellectual, 
psychological, emotional, and social development to take place (National Research Council, 
2002). As mentioned in the Program Environment section of this chapter, students generally 
felt safe in their after school programs. Furthermore, intellectual, psychological, emotional, 
and social development occurs via the interaction between students and caring, supportive 
others (National Research Council, 2002). As stated previously, students felt that they had 
supportive relationships with adults in the after school programs. In addition, their parents 
observed that their children had supportive relationships with their peers. Finally, as noted 
earlier, these supportive relations were one means of creating students’ sense of 


belongingness in the programs. 
Appropriate Structures and Positive Social Norms 


All programs have a set of norms that help shape students’ perception of what is 
appropriate behavior. Norms are particularly salient to development because they “shape 
morals, present ways of relating to others, and provide templates of self-control” (National 
Research Council, 2002). Program staff expressed a high degree of support and 
encouragement for students’ use of pro-social behavior. They stated that they implemented 
rules that strongly emphasized such things as respect for others and avoidance of fights 
(M = 3.61, n= 118). These appropriate structures provide positive social norms for positive 
youth development. 


Opportunities for Skill Building, Support for Efficacy and Mattering 


Furthermore, students are given the opportunity to engage in meaningful participation 
when they participate in activities that are relevant, engaging, and interesting (Austin & 
Duerr, 2005). They should also be given opportunities to be autonomous and responsible that 
are appropriate to their developmental stage (National Research Council, 2002). 


As shown in Table 36, students generally reported that they were able to participate in 
their programs in meaningful ways. They participated in activities that interested them (M = 
3.15, n = 564) and, to a lesser extent, were able to choose things to do in the program (M = 
2.57, n = 560). However, it does appear that students did not have much say with respect to 
rule-setting (M = 1.95, n = 558). The following were comments made by a couple of middle 
school students: 

A little too strict. Like, right now they say that we have to stay in one area and we can't 


like switch our activities. Like, if we don’t want to do arts and crafts anymore we have to 
stay there and we can't go to another [activity] like to do softball or anything like that. 
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Another student added: 


And we used to have a coach who let us do [things]. And sometimes you had to go, like, 
outside to find, like, water or, like, if you wanted soda. But [the new coaches] don’t let 
you do anything. They’re pretty bossy [the new coaches] and, like, we used to go outside 
and they’d like (pause)—First, they told us that we could go outside but we had to come 
back inside in time. But now they don’t let us get out because if you get out you stay out. 
Like we were pets, but we’re not; we’re children (pause) teenagers, children. 


On the other hand it also appears that the students were also afforded the opportunity to 
be responsible and contributing members of the program as they did things in the program to 
be helpful (M = 3.07, n = 563). These experiences created a sense of meaningful participation 
for the students. The following quote is from a student’s remarks during the student focus 
groups: 

I used to have doubt in graduating and stuff and at the program they’ve [the site staff] 

helped me (pause) they’ve helped me, like, feel more comfortable. And they helped me 


raise up my (pause) my feelings for, like, that helped me know that I could graduate. And 
that they helped me do more of my math and English. 


Table 36 


Meaningful Participation: Students' Perspective 


Survey prompt Mean % Strongly agree 
I do activities that interest me. (n = 564) 3.15 41.0 
I am able to choose things to do. (n = 560) 2.57 20.5 
I can help make rules if I want to. (n = 558) 1.95 7.9 
I do things to be helpful. (n = 563) 3.07 35.2 


Similar to the students, both the site coordinators and site staff tended to agree that 
students were able to participate in their programs in meaningful ways. As shown in 
Table 37, they stated that students could give feedback about the activities in which they 
participated and about what activities they would like made available to them. They also 
noted that students were given the opportunity to plan and carry out activities, although to a 
lesser extent than giving feedback. 


74 


Table 37 


Meaningful Participation: Staff Perspective 


Site coordinator Site staff 
% Strongly % Strongly 

Survey prompt n Mean agree n Mean agree 
The staff give students opportunities 18 3.28 50.0 117 3.34 44.4 
to give their input about activities 
they would like to have in the 
program. 
The staff give students opportunities 18 3.06 41.2 118 3.11 32.2 
to plan and carry out activities in the 
program. 
The staff give students opportunities 17 3.41 41.2 117 3.21 38.5 


to provide feedback about the 
activities they are currently doing in 
the program. 


High Expectations 


To encourage student efficacy and mattering, after school site staff should let students 
know that they believe that the students can and will succeed and that they are resilient; they 
should provide guidance that is youth-centered and strengths-focused (Austin & Duerr, 
2005). At these program sites, students generally felt that the site staff had high expectations 
for them. They reported that the staff believed they could do a good job (M = 3.28, n = 561). 
In addition, students reported that staff would make sure to comment upon their successful 
endeavors (M = 3.09, n = 563). 


With respect to staff members’ emphasis on academic success, both site coordinators 
and site staff members generally were in agreement with students, as shown in Table 38. 
Both stakeholders indicated that they place a fairly strong emphasis on academics. However, 
site coordinators tended to have a slightly more positive view of enhancing expectations for 


student academic success than site staff. 
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Table 38 


Program and Staff Members' Expectations for Academic Success 


Site coordinator Site staff 
% Strongly % Strongly 
Survey prompt n Mean agree n Mean agree 
Staff members show students 18 3.56 61.1 118 3.36 40.7 


that they care about them doing 
well academically. 


The program strongly 18 3.67 72.2 119 3.47 53.8 
emphasizes/ teaches students 
that school is important. 


Future Aspirations 


Using caring relationships as a conduit, after school staff may convey high expectations 
for students. It is through this nurturing relationship that students can learn to believe that 
they will be successful in the future (Austin & Duerr, 2005). Similar to staff expectations of 
students’ academic success, staff members stated that they encouraged their students to plan 
for their future (M = 3.59, n = 117), finish high school (M = 3.62, n=117), and attend 
college or vocational school (M = 3.62, n= 116). 


Other Aspects of Positive Youth Development 


The staff survey also inquired about the various aspects of positive youth development 
that the site staff cultivate. Site staff reported that they foster students’ feelings of academic 
self-efficacy (M = 3.52,). In addition, they foster students’ cognitive competence with respect 
to critical thinking (M = 3.36) and problem-solving (M = 3.25)Finally, site staff cultivate 
their students’ socio-emotional competence by encouraging them to be empathetic 
(M = 3.64). 


Integration of Family, School, and Community Efforts 


Finally, positive youth development is facilitated when there is communication 
amongst the various settings in which the youth interact. Consistent positive messages across 
settings decrease the likelihood that youth will adopt deviant values and behavior (National 
Research Council, 2002). As stated previously, there was a fair amount of integration and 
communication between the after school programs, the host schools, the community, and 


parents (see Local Partnerships). One of the school principals commented: 
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One of the most important ones is with Campbell’s Soup, where they donate soup and 
even to family members if they show up twice a week. That was a big deal for parents. 
There’s also great connection with law enforcement, the fire station. Those are a few that 
come to mind...The soup wouldn’t have an academic importance, its more of a social- 
health importance. Really the others that came to mind, they may not have an academic 
focus. It’s more of a community performance. 


Another program director mentioned: 


Because we have all these additional resources with Boys and Girls Club, we are able to 
bring out...for example, we have a partnership with Chalk and Health Smiles, so during 
our family nights they’ll send out a representative. So if there’s parents that have no 
health insurance or need resources, we are able to connect them with this other agency to 
get referred to other programs. 


With these setting features in place, the following section provides the perceived 


outcomes of the students. 
Perceived Outcomes 


It has been suggested that the greater the number of features of positive developmental 
settings a program has, the more significant the impact it will have on the positive 
development of youth (National Research Council, 2002). Several resilience or positive 
youth development traits may be fostered, including academic self-efficacy, academic 
attitudes, academic skills, cognitive competence, socio-emotional competence, school 
attendance, students’ life-skills/knowledge, and future aspirations (Austin & Duerr, 2005; 
Catalano, Berglund, Ryan, Lonczak, & Hawkins, 2004). 


Academic Self-Efficacy 


The CRESST instruments probed students and parents with a series of questions about 
students’ feelings of self-efficacy. Both students and parents expressed that attending a 
program has led students to feel more efficacious with respect to their academic skills. When 
asked whether the program has helped them do better in school, some students responded: 


Student 1: They help with homework and they teach me a little math and English. 
Student 2: They helped me do more of my math and English. 


It should be noted that parents tended to feel more positive about an improvement in 
their children’s confidence in their academic abilities (M = 3.31); whereas, students 
expressed less confidence when asked if their comfort-level during tests had improved 
(M = 2.75). 
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Academic Attitudes 


Academic attitudes have been found to be associated, both directly and indirectly, with 
achievement (Abu-Hilal, 2000; Dumais, 2009). When asked, students expressed that 
attending an after school program resulted in an improvement in their attitude towards 
school. They stated that the program helped them want to come to school more often 
(M = 3.03,). They also said that the program helped them work harder in school (M = 3.04,). 
After school staff and parents also reported that attending a program had a positive effect on 
students’ academic attitudes. Table 39 presents their responses to the survey items regarding 
students’ academic attitudes. 


Table 39 


Academic Attitudes: After School Site staff and Parents' Perspective 


Site coordinator Site staff Parents 
% % % 
Strongly Strongly Strongly 

Survey prompt n Mean agree n Mean agree n Mean agree 
I am satisfied that the 18 3.39 38.9 116 3.31 36.2 879 =. 3.33 41.3 
program has helped 
students to improve their 
schoolwork habits. 
I am satisfied that the 18 3.44 44.4 115 3.27 38.3 - - - 
program has helped 
students to like going to 
school more. 
I am satisfied that the 18 3.50 50.0 117) 3.27 36.8 - - - 
program has helped 
students to want to attend 
day school more 
regularly. 
I am satisfied that the 17 3.18 23.5 116 3.14 29.3 - - - 
program has helped 


students to want to be on 
time for day school more 
often. 


Academic Skills 


Considering the association between academic self-efficacy, attitudes, and 
achievement, CRESST decided to ask students if they felt that their academic skills had 
improved as a result of attending an after school program. As shown in Table 40, students 
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believed that attending an after school program helped them improve their academic skills. 


However, with respect to language arts, computer use, and test-taking skills, they tended to 


state that their skills improved a little rather than a lot. The notable exception to this finding 


is that students had a more positive view conceming their improvement in homework 


(M = 3.37) than they did concerning their improvement in other aspects of academic 


achievement. 


Table 40 


Academic Skills: Students' Perspective 


Survey prompt Mean % A lot 
How much has this after school program helped you in your 2.83 36.3 
English class? n = 562 
How much has this after school program helped you read better? 2.89 37.5 
n= 560 
How much has this after school program helped you write 2.77 33.4 
better? n = 557 
How much has this after school program helped you solve math 3.12 49.6 
problems better? n = 556 
How much has this after school program helped you do better 3.37 60.1 
with your homework? n = 557 
How much has this after school program helped you get better 3.04 44.5 
grades? n = 559 
How much has this after school program helped you learn to use 2.14 20.8 
computers? n = 557 
How much has this after school program helped you do better on 2.74 32.1 


tests? n = 558 


As shown in Table 41, site coordinators, site staff, and parents generally agreed that 


attending a program resulted in an improvement in students’ academic skills. As compared to 


the students’ responses above, site coordinators, site staff, and parents tended to have a more 


positive view of the improvement in academic skills than did the students. 
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Table 41 


Academic Skills: After School Site staff and Parents' Perspective 


Site coordinator Site staff Parents 


% % % 
Strongly Strongly Strongly 
Survey prompt n M agree n M agree n M agree 


I am satisfied that the 18 = 3.39 38.9 118 = 33.15 28.8 882 3.31 41.3 
program has helped 

students to improve their 

language arts skills. 


I am satisfied that the 18 3.11 27.8 116 =. 3.26 36.2 880 3.29 41.6 
program has helped 

students to improve their 

math skills. 


I am satisfied that the 18 3.28 44.4 113 3.00 23.9 874 3.22 35.8 
program has helped 

students to improve their 

science skills. 


I am satisfied that the 16 3.13 18.8 115 3.10 29.6 861 3.21 34.6 
program has helped 

students to improve their 

standardized test scores. 


I am satisfied that the 878 3.29 41.9 
program has helped my 

child improve his/her 

grades. 


Cognitive Competence 


Site coordinators, site staff, and parents generally agreed that attending a program 


resulted in an improvement in students’ cognitive competence, as shown in Table 42. 
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Table 42 


Cognitive Competence 


Site coordinator Site staff Parents 
% % % 
Strongly Strongly Strongly 
Survey prompt n M agree n M agree n M agree 
I am satisfied that the 16 3.44 50.0 118 3.23 29.7 863 3.27 36.3 
program has helped 
students’ problem- 
solving skills (trying 


different solutions until 
one works, etc.). 


I am satisfied that the 17 3.41 47.1 117 3.34 41.0 874 3.32 41.0 
program has made 

students feel more 

comfortable asking their 

teachers for help when 

needed. 


I am satisfied that the 17 3.35 41.2 118 3.34 41.5 870 3.28 37.6 
program has helped 

students develop 

decision-making skills 

(thinking about possible 

consequences before 

making decisions). 


Socio-Emotional Competence 


As shown in Table 43, students generally reported that their socio-emotional 
competency improved as a result of attending a program. Students tended to agree that they 
were better able to get along with others and make new friends. This finding is not surprising 
considering that most site staff stated that they encouraged pro-social behavior during the 
program (see Appropriate Structures and Positive Social Norms). When students were asked 
whether the program has helped them get along better with their peers, they responded: 


Student 1: They help us with our conflict with them. 

Student 2: They help us get along better by working as a team. 
Student 3: We can participate in class more. 

Student 4: And make new friends...Even when you're shy. 
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Table 43 


Socio-Emotional Competence: Students’ Perspective 


Survey prompt Mean % A lot 
How much has this after school program helped you make new 3.26 54.7 
friends? n = 559 
How much has this after school program helped you get into less 2.99 47.0 
trouble at school? n = 551 
How much has this after school program helped you avoid fights?. n 2.91 49.4 
= 553 
How much has this after school program helped you get along with 3.17 49.2 
others? .n = 559 
How much has this after school program helped you understand other 3.08 45.8 


people’s feelings? n = 559 


How much has this after school program helped you work out 3.06 48.7 
problems with your friends? n = 561 


Table 44 displays site coordinators’, staff’s, and parents’ perspectives on the students’ 
socio-emotional competence. Similar to the students, site coordinators, site staff, and parents 
generally agreed that attending a program resulted in an improvement in students’ socio- 
emotional competence. As with many of the other outcomes, site coordinators tended to have 
an equally or more positive view of the improvement overall. The exception involved 
students’ interest in helping people in their community, in which parents (M = 3.29) had 
moderately more positive views than the site coordinators (M = 3.13) and site staff (M = 
3.09). 
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Table 44 


Socio-Emotional Competence: After School Site staff and Parents’ Perspective 


Site coordinator Site staff Parents 


% % % 
Strongly Strongly Strongly 
Survey prompt n M agree n M agree n M agree 


I am satisfied that the 17 3.35 47.1 118 3.33 39.8 858 3.24 34.8 
program has helped 

students’ leadership 

skills. 


I am satisfied that the 16 3.13 25.0 116 3.09 27.6 862 3.29 38.2 
program has helped 

increase students’ 

interest in helping people 

in the community. 


I am satisfied that the 17 3.53 52.9 118 3.30 34.7 870 3.31 37.8 
program has helped 

students be more 

considerate of other 

people’s feelings. 


I am satisfied that the 17 3.41 47.1 118 3.14 31.4 866 3.21 33.8 
program has helped 

students improve their 

ability to handle their 

emotions in an 

appropriate manner 

(stopping and calming 

down when excited or 

upset, etc.). 


I am satisfied that the 17 3.24 29.4 118 3.23 33.1 868 3.23 35.0 
program has helped 

students improve their 

ability to identify their 

emotions. 


I am satisfied that the 17 3.29 41.2 117 3.21 31.6 867 3.25 34.7 
program has helped 

students improve their 

ability to handle 

disagreements with 

others in a positive way. 


Belonging and Attendance 


Students reported that they felt after school programs helped them feel like they 
belonged in school (M = 3.12, n = 564). This finding is similar to the finding that students 
felt a sense of belongingness in the after school programs (see Opportunities to Belong). 
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When asked about their children’s school attendance, parents stated that they were satisfied 
that their children attended day school more regularly (M = 3.56, n = 867) and their children 
were tardy for day school less often (M = 3.16, n = 825). This finding is not surprising due to 
the fact that it has been shown that a sense of belonging is associated with school attendance 
(Sanchez, Colon, & Esparza, 2005). 


Life Skills and Knowledge 


After school programs can be a means to prepare youth for future challenges and 
opportunities (Reisner, White, Russell, & Birmingham, 2004). Students indicated that the 
programs provided them with information about different jobs or careers (M = 2.99, n = 560). 


Future Aspirations 


Having positive future expectations is associated with better social and emotional 
adjustment in school and is a protective factor against negative developmental outcomes 
(Austin & Duerr, 2005; Catalano, et al., 2004). Similar to life skills and knowledge, students 
reported that the after school programs helped them feel they could reach their goals, believe 
that they could go to college or vocational school, and get a good job after finishing school. 
The data representing students’ perspective on future aspirations is presented in Table 45. 
After school staff and parents also stated that they felt that the program helped increase the 
students’ desire to attend college and their belief that they could get a good job. The data 
representing site coordinators’, staff’s, and parents’ perspective on future aspirations is 


presented in Table 46. 


Table 45 


Future Aspirations: Students’ Perspective 


Survey prompt Mean % A Lot 
How much has this after school program helped you believe you can 3.28 58.5 
finish high school? n = 561 
How much has this after school program helped you believe you can 3.30 59.1 
go to college? n = 560 
How much has this after school program helped you believe you will 3.35 61.6 


get a good job after finishing school? n = 558 


How much has this after school program helped you believe that you 3.32 60.2 
can reach your goals? n = 563 
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Table 46 


Future Aspirations: After School Site staff and Parents’ Perspective 


Site coordinator Site staff Parents 
% % % 
Strongly Strongly Strongly 

Survey prompt n M agree n M agree n M agree 
I am satisfied that the 18 3.33 38.9 116 3.29 37.9 852 3.58 61.4 
program has helped 
students to want to 
attend 
college/vocational 
school. 
I am satisfied that the 18 3.39 50.0 116 3.25 35.3 855 3.49 56.8 
program has helped 
students believe they 
will get a good job after 
high school. 
I am satisfied that the 18 3.33 44.4 116 3.18 35.7 
program has helped 


increase student interest 
in certain career fields. 


Study findings revealed that the programs which were visited showed indications of 
embodying the eight features of positive youth development settings. Students felt safe in the 
programs, had positive relationships with both adults and peers, and felt a sense of 
belongingness in the programs and schools. In order for after school programs to maintain 
their efficacious outcomes, they need to set up an evaluative system to monitor their 
successes and failures. The next section discusses the process of continuous program 


improvement. 
Evaluation System and Use of Data for Continuous Improvement 


It is noted by the U.S. Department of Education and U.S. Department of Justice (2000) 
that effective after school programs should use continuous evaluations to fine tune and 
determine whether they are meeting their program goals. These evaluations generally involve 
gathering data from students, teachers, school administrators, staff, and volunteers to 
continuously monitor instructional adherence to and effectiveness of program goals, to 
provide feedback to all stakeholders for program improvement, and to identify the need for 


additional resources such as increased collaboration, staff, or materials. 
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Site coordinators, staff, and parents were asked to provide feedback on the after school 
programs’ practices on tracking stakeholders’ levels of satisfaction. Survey results indicated 
that parents were less likely to report that the after school program kept track of their levels 
of satisfaction than the site coordinator and staff. These findings are presented in Table 47. 


Table 47 
Does the after school program keep track of the level of satisfaction with the 
program? 
Stakeholder % Yes % No 
Site coordinator (n = 17) 70.6 29.4 
Staff (n = 114) 78.1 21.9 
Parent (n = 858) 65.7 34.3 


Of those sites that did keep track of levels of satisfaction, site coordinators stated that 
they sampled all of the most pertinent stakeholders (e.g., students, parents, and after school 
staff). Table 48 shows the percentage of site coordinators and staff that poll the different 
stakeholders for their levels of satisfaction. 


Table 48 
Which Stakeholders Are Asked About Their Levels of Satisfaction: Site Coordinator and Site 
Staff Perspective 
Stakeholder Site coordinator (n = 12) Site staff (n = 89) 
Students 91.7% 71.9% 
Parents 91.7% 79.8% 
After school staff 91.7% 71.9% 
Day school staff 91.7% 53.94% 
Other stakeholder 25.0% 9.0% 


Site coordinators mostly agreed that their programs have internal evaluation procedures 
(M = 3.29), that they track goal attainment to improve students’ academic and non-academic 
outcomes (M= 3.18), and that staff are surveyed to identify needed areas of program 
improvement (M = 3.13). It is less likely that students’ academic achievement (M = 2.76) or 
social skill development (M = 2.82) were evaluated. Encouragingly, over half (58.8%, 
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M = 3.41) of the site coordinators strongly agreed that the evaluation findings were used to 
improve the program. Table 49 displays the findings. 


Table 49 


Site Coordinator Responses on Questions Assessing Their Use of Evaluation Methods in Their Program. 


Survey prompt Mean % Strongly agree 

Our program does measure and track goal attainment to improve 3.18 41.2 
students’ academic outcomes. (n = 17) 

Our program does measure and track goal attainment to improve 3.18 35.3 
students’ non-academic outcomes. (n = 17) 

We have an internal method for evaluating program activities. (n = 17) 3.29 47.1 

The students’ academic achievement is evaluated. (n = 17) 2.76 11.8 

The students’ social skills development is evaluated. (n = 17) 2.82 5.9 

Staff members are surveyed to identify needed areas of program 3.13 31.3 


improvement. (n = 16) 


Evaluation findings are used to improve the program. (n = 17) 3.41 58.8 


Next, the satisfaction outcomes from different stakeholders are reviewed. 
General Satisfaction Outcomes 


Stakeholders’ satisfaction with the program structure, program environment, and 
program implementation are crucial in maintaining program participation and inducing 
positive youth outcomes (Watts, Witt, & King, 2008).Site coordinators, site staff, and parents 
were asked a series of questions regarding their level of satisfaction with the programs’ 
abilities to meet the students’ academic and emotional needs as well as the programs’ 
structure and implementation. As shown in Table 50, most site coordinators, site staff, and 
parents agreed that the programs met students’ needs and that they are satisfied with the 


programs’ structure and implementation. 
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Table 50 


Satisfaction Regarding Students’ Needs and Program Implementation and Structure 


Site coordinator Site staff Parent 
% Strongly % Strongly % Strongly 
Survey prompt Mean agree Mean agree Mean agree 
I am satisfied that the 3.22 38.9 3.09 24.8 3.38 45.2 
program meets students’ (n= 18) (n= 117) (n = 881) 
academic needs. 
I am satisfied that the 3.22 27.8 3.18 31.6 3.33 40.6 
program meets students’ (n= 18) (n= 117) (n = 875) 
emotional needs. 
I am satisfied with the 3.35 47.1 3.15 31.6 3.44 48.6 
activities that are offered (n=17) (n= 117) (n = 880) 
to the students this year. 
Overall, I am satisfied 3.56 55.6 3.37 44.8 3.57 59.7 
with how the program (n= 18) (n= 116) (n = 879) 
runs. 
I am satisfied with the 3.50 55.6 3.38 43.6 3.47 50.8 
security of the program. (n= 18) (n= 117) (n = 880) 
I am satisfied that the 3.83 83.3 3.53 56.0 3.48 50.5 
program is a good (n= 18) (n= 116) (n = 880) 
environment for students 
to build friendships. 


Student Satisfaction 


Eighty percent (n = 551) of the K-9 student participants reported that they would 
recommend the after school program to a friend. Below is an excerpt from an elementary 
school student focus group. In this excerpt, students were asked if they would recommend the 
program to a friend and why. This excerpt captures students’ satisfaction with the activities, 


resources, space, and academic assistance that the program provides. 


Interviewer: Raise your hand if you would recommend this program to a friend...So, 
why would you recommend this program to a friend? What are some 
reasons? Snake? 


Snake: Because it’s fun and...especially on Fridays. 

Interviewer: Especially on Fridays. What do you guys do on Fridays that’s so fun? 
Snake: We watch movies, go on computers, play outside... 

Interviewer: So you get to do a lot of fun stuff on Fridays. Panda? 


Panda: It’s helpful with your homework. 
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Panda: And it can improve your grades. 


As indicated by the responses, in general, parents, site coordinators, staff, and 
students are satisfied with the program and perceive that the programs have had a positive 
impact on the students. Although evidence and literature indicate that many children and 
youth from low-income families would benefit from participating in after school programs, 
many of these children or youth do not participate. This study explored the obstacles from the 
perspectives of site staff and students. 


Perceived Recruitment Obstacles 


Little (2007) argued that the six most common obstacles that hinder students’ 
participation in after school programs are affordability, students wanting to hang out with 
friends instead, lack of transportation, poor program quality, the need to work after school, 


and family factors such as the need to take care of younger siblings. 
Perceived Obstacles for Participation 


In this study, site coordinators and site staff were presented with these circumstances 
and were asked to select those that affected students in their programs. It is interesting to note 
that the majority of site coordinators and site staff did not perceive any obstacles preventing 
students from participating in the program. The two obstacles that were selected most often 


were language barriers and lack of transportation. 


Students were also queried about their perceived obstacles in participation. Since the 
survey went to students who were already participating regularly, it is not surprising that the 
majority of them did not perceive any obstacles. Approximately 68% of students reported 
that they believe that there was nothing that hindered their participation in the after school 
program. However, it is interesting to point out that some students reported that they had 
another activity, and that made it difficult for them to participate in the program. Table 51 
presents the findings. 
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Table 51 


Perceived Obstacles and Impacts 


What makes it difficult for students to Die oa ae aes 
Ce Percent Percent Percent 
None 52.9 43.4 68.4 
Language Barrier 5.9 22.2 4.4 
Program location 5.9 6.1 5.1 
Lack of transportation 23.5 22.2 3.6 
Students must work after school 11.8 5.1 - 
Students must take care of their siblings 11.8 10.1 6.5 
Students go to another school program - - 5.5 
Students do another activity - - 10.5 
Not satisfied with program - - 1.3 
Other 29.4 19.2 11.1 
Summary 


Based on the Sample IV findings, majority of the elementary and middle school 
students who participated in the survey self-reported to attend school and the after school 
program regularly. Over 80% of them also claimed to have received mostly A’s and B’s. The 
most common reasons for their attending the programs were based on the decisions of 
parents and for homework support. 


The majority of the staff at these program sites reported to have some college 
education, a Bachelor’s degree, and an Associate‘s degree. Most of the staff did not have 
teaching credentials, and when asked if they were currently enrolled in a teaching credential 
program, the majority of the site staff reported that were not currently enrolled. 


It is important to mention that the majority of the staff reported to speak English, and 
Spanish, Since over 60% of the students in the study samples were Hispanic/Latinos, this 
additional language skill is an asset for the after school programs in communicating and 
bonding with students and parents. These programs also appeared to be able to maintain a 
stable environment in terms of staff turnover, as the majority of the staff reported that they 
had worked at the same after school program for more than three years. 
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Moreover, these programs appeared to be well-grounded in program orientation, with 
most of the site coordinators reporting that their programs are oriented by clear goals. Most 
of the program sites also reported to employ a collaborative management style. 


In terms of the programs’ partnerships with external connection, survey instruments 
examined the programs’ relationships with the host schools, parents, and local communities. 
In general, the site coordinators reported a strong connection with the host schools. With 
regards to relationships with local communities, there appeared to be an established system 
for the programs where communities would participate in program events and gave input. 
However, the involvement was less pronounced in terms of local communities providing 
supplies or in terms of local communities being considered during curricular decision- 


making. 


Findings from the parent survey present a picture of a modest degree of involvement: 
60% of parents had visited their children’s program, and about one in three had attended an 
event hosted by their children’s programs. About one in five said they had volunteered or 


given feedback to the programs. 


The most frequently mentioned obstacles to parental involvement include time conflicts 
with jobs, needing to care for other children, and language barrier. It is therefore assumed 
that, in order to increase parental participation, the programs may need to provide childcare, 


and translation services. 


As for program environment, students perceived the program climate to be safe 
physically and emotionally. They felt cared for and supported by the staff, while the staff 
reported to encourage students’ efficacy and aspiration. In terms of resources, in general, 
staff reported to have sufficient writing materials, space to work, and enough time to work 
with the students. However, there was a discrepancy in the levels of agreement between site 
coordinators and site staff about overall resources, with the site coordinators generally 
offering a more positive view. It is also important to note that both site staff and site 
coordinators were likely to report that they did not have time to prepare for activities. 


Regarding instructional features, homework seemed to be the most common activity 
reported by students. Staff were generally adequately prepared for their roles and staff 
efficacy was enhanced by professional development. Both site coordinators and site staff 
agreed that the professional development offered was aligned with program goals and that 
the training was provided annually. Once again, there was a discrepancy in levels of 
agreement between the two stakeholders. It appears that site coordinators generally had a 


more positive view in the training offered compared to the staff’s view in the training they 
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received. In regards to future training, site staff reported that they would like to receive 
training in areas such as behavior management, using age-appropriate teaching techniques, 


teaching academic skills, and explaining ideas to help students to improve understanding. 


Study findings also revealed that the programs which were visited showed indications 
of embodying the eight features of positive youth development settings. Students felt safe in 
the programs, had positive relationships with both adults and their peers, and felt a sense of 
belongingness in the programs and schools. With the exception of rule-setting, students were 
provided with opportunities to participate in activities that were relevant and interesting. 
They also had a fair degree of autonomy within the programs. Finally, site staff and parents 
reported that the programs were moderately integrated with the host schools and the 
communities in which they were located. Parents also played a minor role in the 


implementation of the programs. 


Overall, students, parents, and after school staff all have positive perceptions on the 
effects of program participation. Both students and their parents felt that attending a program 
led to students feeling more academically efficacious. Students, their parents, and site staff 
felt that attending a program had a positive effect on students’ academic attitudes. Students 
felt there was an improvement in their academic skills. However, they were slightly less 
positive in this assertion when compared to their feelings about their growth in efficacy and 
attitudes. Site staff and parents generally agreed that attending a program resulted in an 
improvement in students’ cognitive competence. Students expressed a similar opinion with 
respect to their socio-emotional competence. Parents reported that they believed that 
students’ attendance improved as a result of attending the program. Finally, students reported 
that they felt that they had obtained information about different jobs or careers as a result of 
attending an after school program and helped them to aspire for their future, such as 
attending college. 


In order to maintain and sustain these positive outcomes, after school programs have to 
continuously monitor, evaluate, and fine tune their program activities and offerings. Most site 
coordinators and staff reported that they keep track of the stakeholders’ levels of satisfaction 
with the program. Of these stakeholders, parents were less likely than site coordinators or 
staff to report that the after school program monitors their level of satisfaction. Most site staff 
reported having an internal method of evaluating program activities and using their findings 
to improve the program. A future implication is to identify whether and how the findings that 
the programs obtain are utilized and what outcomes are derived from their monitoring and 


evaluations. Lastly, additional efforts can be made to sample parents, as there was a 
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discrepancy between their perception of the use of evaluation and the site staff and site 
coordinators’ report on how much they were targeted. 


According to the self-reports, parents, site coordinators, and staff agreed that they were 
satisfied across different aspects of the after school programs—specifically, program 
structure, implementation, academic assistance, and meeting students’ emotional needs. 
According to site staff, there were few to none perceived obstacles in student recruitment, the 
two obstacles mostly mentioned were language barriers and lack of transportation. Similarly, 
the majority of students reported that there was nothing that made it difficult for them to 
participate in the program other than the interference of another activity or after school 
program. 


The findings presented in this chapter are preliminary and descriptive in nature. During 
next year (2011) following the year 3 data collection procedures, these data will be merged 
with the new data set and examined longitudinally. Additionally the findings of the four 
study samples will be triangulated and cross-examined to extract common themes and 
patterns. More sophisticated analyses drawing on HLM and using the site/school as a unit of 
analysis are also planned for next year. 


93 


CHAPTER VII: 
SUMMARY OF YEAR 2 FINDINGS 


In Year 2, CRESST focused on data collection. Specifically, the following tasks have 


been accomplished: 
e Study Sample I. Continuous monitoring of the database. 


e Study Sample II. Completed data collection for years 2006-07 through 2008-09 
and analyzed academic performance, attendance, and suspension outcome data. 


e Study Sample III. The After School Profile Questionnaire (Part A and Part B) 
Year 2 data for the ASES and 21% CCLC programs were gathered, and analyses 
were conducted. In addition, the After School Profiling System was designed and 
implemented with the intention that the system may continue to serve as a program- 
monitoring tool for the CDE after CRESST turns over the database to the CDE at 
the end of the study. 


e Study Sample IV. Piloting and the first round of data collection were conducted. 
Research instruments were revised for the remainder of this study. A preliminary 
analysis focused on program features and stakeholder satisfaction for the Study 
Sample IV was conducted. 

Some preliminary analyses were conducted. CRESST urges caution in the 

interpretation of these early findings. For the purpose of this annual report, preliminary 


findings from Year 2 are summarized as follows. 


In Samples I and II, preliminary study findings revealed that after school participation 
at both elementary and middle school levels had minor negative to no effect on students’ 
academic achievement outcomes and small positive effects on some behavioral outcomes. 
There are small positive effects of after school participation on physical fitness and on school 
day attendance rates. For most outcomes, the after school program effects were slightly 
larger for students who frequently attended an after school program, rather than just attending 
at some time during the year. Compared to the results found for all after school participants, 
frequent participants had higher mathematics CST scores for both 2007-08 and 2008-09, 
higher CELDT scores in 2008-09, and larger positive effects on three physical fitness 


measures. 


In Sample III, most of the 396 grantees served by the CDE targeted students considered 
academically at-risk, followed by English learners. Math and language arts were offered by 
most programs. Transportation was sometimes an obstacle to student attendance in rural 
areas. In terms of program structure, many programs funded by the ASES and 21° CCLC 


programs maintained local partnerships, the majority of which were represented by Local 
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Education Agencies of public schools. Internally, after school programs in this sample tended 
to use a collaborative management style. Staff qualifications varied depending on the role 
and location of a site, while turnover rates were lower for credentialed teachers and site 
leaders than for paraprofessionals and/or instructional aides. Professional development 


opportunities were mainly offered by the after school program. 


In Sample IV, CRESST utilized a comprehensive set of instruments to study 13 
randomly selected programs. Data drawn from this phase of the research comes from a wide 
range of after school program stakeholders: day school administrators, program directors, site 
coordinators, site staff, parents, students, and the research team’s classroom observations. 
The program orientation, program environment, instructional features, evaluation system 
(program features) and its effects on positive youth development, and stakeholder satisfaction 
(program outcomes) were evaluated. CRESST also examined other perceived obstacles and 
impacts faced by the ASES and 21“ CCLC grantees. 


Preliminary findings are promising. Overall, students felt safe in the programs, had 
positive relationships with both adults and their peers, and felt a sense of belongingness in 
the programs. Both students and their parents felt that attending a program led to students 
feeling more academically efficacious. Despite expressing different degrees of satisfaction, 
parents, site coordinators, and staff generally agreed that they were satisfied across multiple 
aspects of the after school programs, including program structure, implementation, and their 
effects on students’ participation. The lack of transportation, language barriers, and the need 
to take care of siblings or to work after school are the barriers perceived by staff in terms of 


student participation and parental involvement in the programs. 
Looking Ahead: Roadmap for Year 3 


The in-depth data collection on the subsample (Sample IV) will continue during Year 3. 
The ASES sites will be revisited between October 15, 2010 and March 30, 2011. CRESST 
will again revisit the contact list for all ASES and 21° CCLC grantees. The After School 
Profiling Questionnaire will be rolled out again in mid-January to update the profiles of the 
grantees. The system is anticipated to close down on April 15, 2010. Year 2 findings will 
support the ongoing development of the profiling system and provide the necessary 
information for the grantee database to be monitored and merged with data from the previous 


year (Year 1) and the incoming year (Year 3). 


CRESST will also extend the analyses in Samples I and II to incorporate additional 


behavioral outcomes—student mobility and classroom behavior—and examine variation in 
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after school participation effects across schools. The longitudinal nature of after school 
participation and participation effects for specific student cohorts will also be examined. 


Renewal of the application for UCLA OPRS will be a continuous effort as instruments 
are revised and new procedures developed. Annual reports will be provided at the end of 
Year 3. 
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APPENDIX A: 
SUMMARY OF DATA SOURCES AND PROFILES OF STUDENT 
CHARACTERISTICS (CHAPTER IIT) 


Table Al 


Profile of ASES/21* Century (Grades 2-8) Participants by Sample III Inclusion: 
Student Characteristics 


Total 2008-09 2009-10 
inSTAR SampleI Sample III Sample III 


Number of Students 386,298 380,410 190,760 139,808 
% Female 49% 49% 49% 49% 
Race/Ethnicity (%): 
African American/Black 11% 11% 10% 11% 
Asian/Pacific Islander 8% 8% 8% 10% 
Hispanic/Latino 67% 67% 71% 66% 
White 13% 13% 10% 12% 
Other 2% 2% 1% 2% 


Eng. Lang. Class. (%): 


English Only 40% 40% 36% 38% 
I-FEP 7% 7% 8% 7% 
R-FEP 11% 11% 12% 11% 
English Learner 41% 42% 44% 43% 
Parent Education (%): 
College Degree 12% 12% 9% 10% 
Some College 16% 16% 15% 15% 
High School Graduate 23% 23% 24% 23% 
Less Than HS Grad 25% 25% 27% 25% 
Non-Response 24% 24% 25% 28% 
% Title I 81% 82% 85% 81% 
% NSLP 79% 79% 83% 80% 
% Student w/Disabilities 10% 10% 10% 10% 
% GATE 8% 8% 7% 8% 
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Table A1 Continued 


Profile of ASES/21* Century (Grades 2-8) Participants by Sample III Inclusion: 
Student Characteristics 


Total 2008-09 2009-10 
inSTAR Sample I Sample III Sample III 


% New to School 39% 38% 38% 38% 

% Retained in Grade 1% 1% 1% 1% 

Grade Level (%): 
2nd Grade 14% 14% 15% 15% 
3rd Grade 16% 16% 16% 17% 
4th Grade 15% 15% 15% 16% 
5th Grade 14% 14% 14% 15% 
6th Grade 15% 15% 15% 13% 
7th Grade 13% 13% 13% 13% 
8th Grade 12% 12% 12% 12% 


2007 CST ELA Results: 
% Prof. or Advanced 31% 31% 29% 30% 
Standardized Scale Score -0.33 -0.33 -0.38 -0.35 


2007 CST Math Results: 


% Prof. or Advanced 40% 40% 39% 40% 
Standardized Scale Score -0.26 -0.26 -0.29 -0.27 
Table A2 


Profile of ASES/21* Century (Grades 2-8) Participants by Sample III Inclusion: 
School Characteristics 


Total 2008-09 2009-10 
inSTAR SampleI Sample III Sample III 


Mean School Enrollment 742 745 811 685 
Mean 2007 Base API 705 704 696 703 
Mean School SCI 163 163 162 163 


School Location (%): 


Urban 59% 60% 63% 64% 
Suburban 39% 39% 35% 34% 
Rural 6% 6% 6% 7% 
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Table A3 


Profile of ASES/21* Century (Grades 2-8) Participants by Sample IV Sites Included in 
Analysis: Student Characteristics 


Total Original Included 
inSTAR SampleI SampleII Sample IV Sample IV 


Number of Students 386,298 380,410 286,067 5,629 4,251 
% Female 49% 49% 49% 49% 49% 
Race/Ethnicity (%): 
African American/Black 11% 11% 12% 11% 7% 
Asian/Pacific Islander 8% 8% 9% 7% 7% 
Hispanic/Latino 67% 67% 68% 76% 84% 
White 13% 13% 10% 5% 2% 
Other 2% 2% 1% 1% 0% 


Eng. Lang. Class. (%): 


English Only 40% 40% 38% 30% 21% 
I-FEP 7% 7% 8% 8% 9% 
R-FEP 11% 11% 12% 16% 21% 
English Learner 41% 42% 42% 46% 49% 


Parent Education (%): 


College Degree 12% 12% 11% 8% 6% 
Some College 16% 16% 15% 13% 10% 
High School Graduate 23% 23% 22% 22% 20% 
Less Than HS Grad 25% 25% 24% 31% 29% 
Non-Response 24% 24% 27% 26% 36% 
% Title I 81% 82% 83% 85% 98% 
% NSLP 79% 79% 80% 84% 88% 
% Student w/Disabilities 10% 10% 10% 10% 10% 
% GATE 8% 8% 8% 5% 7% 
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Table A3 Continued 


Profile of ASES/21* Century (Grades 2-8) Participants by Sample IV Sites Included in 
Analysis: Student Characteristics 


Total Original Included 
inSTAR SampleI Sample II Sample IV Sample IV 
% New to School 39% 38% 38% 39% 34% 
% Retained in Grade 1% 1% 1% 1% 1% 
Grade Level (%): 
2nd Grade 14% 14% 14% 13% 11% 
3rd Grade 16% 16% 15% 15% 13% 
Ath Grade 15% 15% 15% 14% 11% 
5th Grade 14% 14% 13% 12% 13% 
6th Grade 15% 15% 15% 17% 20% 
7th Grade 13% 13% 14% 14% 16% 
8th Grade 12% 12% 13% 14% 17% 


2007 CST ELA Results: 
% Prof. or Advanced 31% 31% 31% 23% 22% 
Standardized Scale Score -0.33 -0.33 -0.33 -0.49 -0.52 


2007 CST Math Results: 


% Prof. or Advanced 40% 40% 40% 33% 32% 
Standardized Scale Score -0.26 -0.26 -0.26 -0.42 -0.41 
Table A4 


Profile of ASES/21* Century (Grades 2-8) Participants by Sample IV Sites Included in 
Analysis: School Characteristics 


Total Original Included 
inSTAR SampleI SampleII Sample IV Sample IV 


Mean School Enrollment 742 745 834 1,039 1,214 
Mean 2007 Base API 705 704 699 676 657 
Mean School SCI 163 163 163 160 159 


School Location (%): 


Urban 59% 60% 69% 78% 70% 
Suburban 39% 39% 33% 20% 30% 
Rural 6% 6% 1% 2% 0% 
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Table A5 


Summary of Data Collection 


Planned data coverage 


Database Source Population Time period Current status 
After school The CDE All AfterSchool 2006-07to Received & processed data for 2006-07 
attendance Participants 2009-10 and 2007-08 
STAR The CDE All tested 2005-06 to Received & processed data for 2005-06, 
students (grades 2009-10 2006-07, and 2007-08 
2-9) 
CELDT The CDE All tested 2005-06 to Received & processed data for 2005-06, 
students (grades 2009-10 2006-07 and 2007-08 
K-9) 
CSIS The CDE All students 2006-07 to Received data for 2005-06, 2006-07, and 
2009-10 2007-08; Data processing in progress 
Student behavior LEA All students in 2006-07 to — Data collection protocol in development; 
& performance Sample II 2009-10 expect to contact Sample II districts in 
data October 2009 
After school CRESST ~All AfterSchool  2008-09to — Data collection completed; Analyses for 
profile Agencies & Sites 2010-11 2008-09 and 2009-10 have been 
questionnaire completed 
Principal CRESST Schools in 2009-10 to — Protocol approved and is in use 
interview Sample IV 2010-11 
Project director CRESST Schools in 2009-10 to — Protocol approved and is in use 
interview Sample IV 2010-11 
Site coordinator CRESST Schools in 2010-11 Protocol created and approved for use in 
survey Sample IV Fall 2010 
Site staff survey CRESST — Schools in 2009-10 to — Protocol approved and is in use 
Sample IV 2010-11 
Site coordinator CRESST — Schools in 2009-10 to — Protocol approved and is in use 
survey Sample IV 2010-11 
Parent survey CRESST Schools in 2009-10 to — Protocol approved and is in use 
Sample IV 2010-11 
Student survey CRESST Schools in 2009-10 to — Protocol approved and is in use 
Sample IV 2010-11 
Student focus CRESST Schools in 2009-10 to — Protocol approved and is in use 
groups Sample IV 2010-11 
Site staff focus CRESST Schools in 2009-10 to — Protocol approved and is in use 
groups Sample IV 2010-11 
Site observations CRESST — Schools in 2009-10 to — Protocol approved and is in use 
Sample IV 2010-11 


Note. CELDT = California English Language Development Test; CSIS = California School Information 
Services; LEA = Local Educational Agency; STAR = Standardized Testing and Reporting. 
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Table A6 
Profile of ASES/21st CCLC (Grades 2-8) Participants by Study Sample (Student Characteristics) 


Total 
in STAR Sample I Sample II SampleIII_ Sample IV 

Number of Students 386,298 380,410 263,470 190,760 5,629 
Number of Schools 5,034 3,053 1,812 1,593 40 
Number of Districts 700 415 100 238 24 
Number of Counties 54 54 32 42 13 
% Female 49% 49% 49% 49% 49% 
Race/Ethnicity (%): 

African American/Black 11% 11% 13% 10% 11% 

Asian/Pacific Islander 8% 8% 9% 8% 7% 

Hispanic/Latino 67% 67% 68% 71% 76% 

White 13% 13% 10% 10% 5% 

Other 2% 2% 1% 1% 1% 
Eng. Lang. Class. (%): 

English Only 40% 40% 37% 36% 30% 

I-FEP 7% 7% 8% 8% 8% 

R-FEP 11% 11% 13% 12% 16% 

English Learner 41% 42% 42% 44% 46% 
Parent Education (%): 

College Degree 12% 12% 11% 9% 8% 

Some College 16% 16% 14% 15% 13% 

High School Graduate 23% 23% 22% 24% 22% 

Less Than HS Grad 25% 25% 24% 27% 31% 

Non-Response 24% 24% 28% 25% 26% 
% Title I 81% 82% 83% 85% 85% 
% NSLP 79% 79% 79% 83% 84% 
% Student w/Disabilities 10% 10% 10% 10% 10% 
% GATE 8% 8% 9% 7% 5% 
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Table A6 Continued 
Profile of ASES/21st CCLC (Grades 2-8) Participants by Study Sample (Student Characteristics) 


Total 
in STAR Sample I Sample II Sample III _ Sample IV 


% New to School 39% 38% 38% 38% 39% 
% Retained in Grade 1% 1% 1% 1% 1% 


Grade Level (%): 


2nd Grade 14% 14% 14% 15% 13% 
3rd Grade 16% 16% 16% 16% 15% 
4th Grade 15% 15% 15% 15% 14% 
5th Grade 14% 14% 14% 14% 12% 
6th Grade 15% 15% 15% 15% 17% 
7th Grade 13% 13% 13% 13% 14% 
8th Grade 12% 12% 13% 12% 14% 


2007 CST ELA Results: 
% Prof. or Advanced 31% 31% 31% 29% 23% 
Standardized Scale Score -0.33 -0.33 -33% -0.38 -0.49 


2007 CST Math Results: 
% Prof. or Advanced 40% 40% 40% 39% 33% 
Standardized Scale Score -0.26 -0.26 -0.26 -0.29 -0.42 


Note. Sources: 2006-07 and 2007-08 STAR data files matched with 2007-08 after school 
attendance data. “New to School” means student CDS code in 2008 does not match CDS code in 
2007; “Retained in Grade” means student grade-level in 2008 is the same (or lower) than the 
grade-level reported in 2007; “Standardized Scale Scores” are the CST scale scores converted to 
z-scores (M = 0, std. dev. = 1) based on the statewide mean and standard deviation for each CST 
subject area test. 
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Table A7 
Profile of ASES/21st CCLC (Grades 2-8) Participants by Study Sample (School Characteristics) 


Total 
in STAR Sample I Sample II Sample III _ Sample IV 

Number of Schools 5,034 3,053 1,812 1,593 40 
Mean School Enrollment 742 745 842 811 1,039 
Mean 2007 Base API 705 704 701 696 676 
Mean School SCI 163 163 163 162 160 
School Location (%): 

Urban 59% 60% 67% 63% 78% 

Suburban 39% 39% 34% 35% 20% 

Rural 6% 6% 2% 6% 2% 


Note. Sources: The CDE Public School Database and 2007 Base Academic Performance Index file. 
API = Academic Performance Index; SCI = School performance index. 
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Table A8 


Goals Set During the 2009-10 School Year by Urbanicity, Part A Questionnaire 


Goals set City Suburb Town/rural Overall Total 
Academic improvement (n= 715) (n = 374) (n = 222) (n= 1,311) 
Goal set 93.4% 98.1% 84.7% 93.3% 
Goal not set 6.6% 1.9% 15.3% 6.7% 
Homework completion (n = 696) (n = 325) (n = 209) (n =1,230) 
Goal set 81.0% 87.4% 71.8% 81.1% 
Goal not set 19.0% 12.6% 28.2% 18.9% 
Improved program attendance (n = 640) (n =362) (n= 201) (n = 1,203) 
Goal set 54.2% 85.6% 62.7% 65.1% 
Goal not set 45.8% 14.4% 37.3% 34.9% 
Improved day school attendance (n = 629) (n = 316) (n = 188) (n = 1,133) 
Goal set 47.2% 68.4% 53.7% 54.2% 
Goal not set 52.8% 31.6% 46.3% 45.8% 
Positive behavior change (n = 590) (n = 323) (n = 198) (n = 1,111) 
Goal set 58.3% 86.1% 95.5% 73.0% 
Goal not set 41.7% 13.9% 4.5% 27.0% 
Increased skill development (n = 566) (n = 311) (n = 183) (n = 1,060) 
Goal set 50.9% 77.8% 44.8% 57.7% 
Goal not set 49.1% 22.2% 55.2% 42.3% 
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Table A9 


Goals Set During the 2009-10 School Year by Grade Span, Part A Questionnaire 


Goals set Elementary School Middle School Overall Total 
Academic improvement (n = 1,001) (n = 303) (n = 1,311) 
Goal set 94.6% 88.8% 93.3% 
Goal not set 5.4% 11.2% 6.7% 
Homework completion (n = 944) (n = 281) (n = 1,230) 
Goal set 82.8% 75.1% 81.1% 
Goal not set 17.2% 24.9% 18.9% 
Improved program attendance (n = 929) (n = 270) (n = 1,203) 
Goal set 63.4% 71.1% 65.1% 
Goal not set 36.6% 28.9% 34.9% 
Improved day school attendance (n = 875) (n = 253) (n = 1,133) 
Goal set 52.0% 61.7% 54.2% 
Goal not set 48.0% 38.3% 45.8% 
Positive behavior change (n = 851) (n = 256) (n= 1,111) 
Goal set 70.4% 81.6% 73.0% 
Goal not set 29.6% 18.4% 27.0% 
Increased skill development (n = 822) (n = 234) (n = 1,060) 
Goal set 56.9% 60.7% 57.7% 
Goal not set 43.1% 39.3% 42.3% 
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Table A10 


Evaluation of Goals and Progress Made During the 2009-10 School Year by Urbanicity, Part A Questionnaire 


Goals and progress made City Suburb Town/rural Overall Total 
Academic improvement (n = 668) (n = 367) (n = 188) (n = 1,223) 
Goal not evaluated 26.9% 35.4% 29.8% 29.9% 
Met or progressed towards goal 73.1% 64.6% 70.2% 70.1% 
Homework completion (n = 564) (n = 284) (n = 150) (n = 998) 
Goal not evaluated 29.4% 41.9% 39.3% 34.5% 
Met or progressed towards goal 70.6% 58.1% 58.7% 65.2% 
Improved program attendance (n = 347) (n = 310) (n= 126) (n = 783) 
Goal not evaluated 42.4% 31.9% 27.8% 35.9% 
Met or progressed towards goal 57.6% 68.1% 72.2% 64.1% 
Improved day school attendance (n = 297) (n= 216) (n= 101) (n= 614) 
Goal not evaluated 30.3% 31.0% 28.7% 30.3% 
Met or progressed towards goal 68.0% 68.1% 71.3% 68.6% 
Positive behavior change (n = 344) (n = 278) (n= 189) (n= 811) 
Goal not evaluated 28.5% 24.1% 20.6% 25.2% 
Met or progressed towards goal 71.5% 75.2% 79.4% 74.6% 
Increased skill development (n = 288) (n = 242) (n = 82) (n= 612) 
Goal not evaluated 41.7% 45.5% 36.6% 42.5% 
Met or progressed towards goal 58.3% 54.5% 63.4% 57.5% 
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Table A11 


Evaluation of Goals and Progress Made During the 2009-10 School Year by Grade Span, Part A Questionnaire 


Goals and progress made Elementary School Middle School Overall Total 
Academic improvement (n = 947) (n = 269) (n = 1,223) 
Goal not evaluated 30.9% 26.4% 29.9% 
Met or progressed towards goal 69.1% 73.6% 70.1% 
Homework completion (n = 782) (n= 211) (n = 998) 
Goal not evaluated 34.7% 34.6% 34.5% 
Met or progressed towards goal 65.1% 64.9% 65.2% 
Improved program attendance (n = 589) (n = 192) (n = 783) 
Goal not evaluated 39.0% 26.6% 35.9% 
Met or progressed towards goal 61.0% 73.4% 64.1% 
Improved day school attendance (n = 455) (n= 156) (n= 614) 
Goal not evaluated 34.5% 18.6% 30.3% 
Met or progressed towards goal 64.6% 79.5% 68.6% 
Positive behavior change (n = 599) (n = 209) (n= 811) 
Goal not evaluated 27.2% 19.6% 25.2% 
Met or progressed towards goal 72.5% 80.4% 74.6% 
Increased skill development (n = 468) (n = 142) (n= 612) 
Goal not evaluated 47.4% 26.8% 42.5% 
Met or progressed towards goal 52.6% 73.2% 57.5% 
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Table A12 


Programming Emphasis by Urbanicity, Part B Questionnaire 


Emphasis City Suburb Town/rural Total 
Academic achievement (n = 692) (n = 363) (n = 232) (n = 1,287) 
A great deal 86.1% 88.2% 84.9% 86.5% 
Somewhat or less 13.9% 11.8% 15.1% 13.5% 
Homework assistance (n = 692) (n = 364) (n = 232) (n = 1,288) 
A great deal 92.5% 91.2% 90.5% 91.8% 
Somewhat or less 7.9% 8.8% 9.5% 8.2% 
Tutoring (n = 674) (n = 358) (n = 229) (n = 1,261) 
A great deal 50.1% 47.8% 60.7% 51.4% 
Somewhat or less 49.9% 52.2% 39.3% 48.6% 
Non-academic enrichment (n = 682) (n = 362) (n= 231) (n = 1,275) 
A great deal 70.2% 70.4% 63.2% 69.0% 
Somewhat or less 29.8% 29.6% 36.8% 31.0% 
Program attendance (n = 692) (n = 364) (n = 230) (n = 1,286) 
A great deal 87.0% 87.9% 70.4% 84.3% 
Somewhat or less 13.0% 12.1% 29.6% 15.7% 
School attendance (n = 680) (n = 361) (n = 228) (n = 1,269) 
A great deal 65.7% 68.7% 53.1% 64.3% 
Somewhat or less 34.3% 31.3% 46.9% 35.7% 
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Table A13 


Programming Emphasis by Grade Span, Part B Questionnaire 


Emphasis Elementary School Middle School Total 
Academic achievement (n = 980) (n = 300) (n = 1,287) 
A great deal 86.6% 86.0% 86.5% 
Somewhat or less 13.4% 14.0% 13.5% 
Homework assistance (n = 981) (n = 300) (n = 1,288) 
A great deal 91.5% 92.7% 91.8% 
Somewhat or less 8.5% 7.3% 8.2% 
Tutoring (n = 955) (n = 299) (n = 1,261) 
A great deal 46.9% 65.2% 51.4% 
Somewhat or less 53.1% 34.8% 48.6% 
Non-academic enrichment (n = 968) (n = 300) (n = 1,275) 
A great deal 68.5% 71.0% 69.0% 
Somewhat or less 31.5% 29.0% 31.0% 
Program attendance (n = 980) (n = 299) (n = 1,286) 
A great deal 86.5% 76.9% 84.3% 
Somewhat or less 13.5% 23.1% 15.7% 
School attendance (n = 964) (n = 298) (n = 1,269) 
A great deal 65.7% 59.4% 64.3% 
Somewhat or less 34.3% 40.6% 35.7% 


Table A14 


Alignment between Goals Set, Progress Made, and Program Emphasis, Part A and B Questionnaires 


Academic Homework Program Day school 
achievement (n = 923) attendance attendance 
(n = 1,040) (n = 539) (n = 483) 
Goal not set, evaluated or 2.0% 4.2% 3.5% 4.8% 
failed to progress 
Met or progressed towards 98.0% 95.8% 96.5% 95.2% 
goal 


Note. Program emphasis represents the percentage of sites that reported they emphasized a programmatic 
feature “A great deal” only. 
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Table A15 
Core Academic Activities Offered during the 2009-10 School Year by Urbanicity, Part B Questionnaire 


City Suburb Town/rural Total 
Core academic activity (n = 724) (n = 376) (n = 236) (n = 1,336) 
History 44.2% 42.3% 39.4% 42.8% 
Language arts/literacy 80.2% 76.9% 78.0% 78.9% 
Math 81.5% 79.0% 85.2% 81.4% 
Science 67.4% 64.9% 59.7% 65.3% 


Table A16 
Core Academic Activities offered during the 2009-10 School Year by Grade Span, Part B Questionnaire 


Elementary School Middle School Total 
Core academic activity (n =1,017) (n = 312) (n = 1,336) 
History 42.9% 42.9% 42.8% 
Language arts 81.3% 70.8% 78.9% 
Math 83.0% 76.0% 81.4% 
Science 69.3% 52.2% 65.3% 


Table A17 


General Academic Assistance Activities offered during the 2009-10 School Year by Urbanicity, Part B 
Questionnaire 


City Suburb Town/rural Total 

General academic activity (n = 724) (n = 376) (n = 236) (n = 1,336) 
Academic enrichment 89.6% 90.2% 89.0% 89.7% 
Career technical education 14.0% 13.8% 18.9% 13.0% 
College preparation 16.2% 15.7% 5.5% 14.1% 
Computer programming/IT skills 36.7% 37.2% 41.1% 37.6% 
Entrepreneurship 11.7% 8.2% 5.5% 9.7% 
Expanded library services 22.1% 22.6% 32.2% 24.0% 
Homework assistance 94.3% 95.7% 95.3% 94.9% 
Mentoring programs 36.5% 31.9% 32.6% 34.5% 
Nutrition education 71.4% 73.4% 69.5% 71.6% 
Preparation for the CAHSEE 3.3% 5.9% 3.0% 4.0% 
Remedial education 14.4% 10.9% 23.7% 15.0% 
Tutoring 58.6% 57.2% 72.9% 60.7% 


115 


Table A18 


General Academic Assistance Activities offered during the 2009-10 School Year by Grade Span, Part B 
Questionnaire 


Elementary School Middle School Total 
General academic assistance (n =1,017) (n = 312) (n = 1,336) 
Academic enrichment 90.0% 88.5% 89.7% 
Career technical education 10.9% 19.9% 13.0% 
College preparation 8.7% 32.1% 14.1% 
Computer programming/IT skills 35.6% 43.3% 37.6% 
Entrepreneurship 8.6% 12.8% 9.7% 
Expanded library services 22.0% 29.8% 24.0% 
Homework assistance 95.1% 94.2% 94.9% 
Mentoring programs 31.5% 43.3% 34.5% 
Nutrition education 74.4% 62.2% 71.6% 
Preparation for the CAHSEE 3.2% 6.4% 4.0% 
Remedial education 12.9% 22.4% 15.0% 
Tutoring 56.4% 74.4% 60.7% 


Table A19 


Developmental Non-Academic Activities offered during the 2009-10 School Year by Urbanicity, Part B 
Questionnaire 


Developmental non-academic City Suburb Town/rural Total 
activities (n = 724) (n = 376) (n = 236) (n = 1,336) 

Career development 17.8% 19.1% 10.6% 16.9% 
Community service 44.9% 56.4% 46.2% 48.4% 
Counseling/character education 30.2% 44.1% 36.4% 35.3% 
programs 
Leadership/entrepreneurial skills 45.6% 49.5% 39.8% 45.7% 
development 
Mentoring opportunities 31.5% 35.4% 31.4% 32.6% 
School safety 56.2% 60.9% 53.8% 57.1% 
Service learning 31.5% 33.0% 31.8% 32.0% 
Tutoring younger pupils 43.4% 48.9% 54.2% 46.9% 
Youth development 56.2% 53.2% 46.6% 53.7% 
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Table A20 


Developmental Non-Academic Activities offered during the 2009-10 School Year by Grade Span, Part B 
Questionnaire 


Developmental non-academic Elementary School Middle School Total 
activities (n =1,017) (n = 312) (n = 1,336) 
Career development 14.6% 25.0% 16.9% 
Community service 46.6% 53.8% 48.4% 
Counseling/character education 32.8% 43.3% 35.3% 
programs 
Leadership/ entrepreneurial skills 43.2% 53.5% 45.7% 
development 
Mentoring opportunities 29.4% 42.6% 32.6% 
School safety 61.0% 44.9% 57.1% 
Service learning 31.0% 34.9% 32.0% 
Tutoring younger pupils 46.8% 46.2% 46.9% 
Youth development 53.2% 55.1% 53.7% 
Table A21 


Common Non-Academic Activities offered during the 2009-10 School Year by Urbanicity, Part B 
Questionnaire 


City Suburb Town/rural Total 
Common non-academic activity (n = 724) (n = 376) (n = 236) (n = 1,336) 
Arts/music 89.1% 91.5% 87.7% 89.5% 
Computer/Internet skills 50.3% 56.1% 65.3% 54.6% 
Coordinated school health services 16.4% 21.3% 14.0% 17.4% 
Physical fitness/sports 91.4% 93.9% 91.5% 92.1% 
Recreational activities 88.7% 88.3% 82.2% 87.4% 
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Table A22 


Common Non-Academic Activities offered during the 2009-10 School Year by Grade Span, Part B 
Questionnaire 


Elementary School Middle School Total 
Common non-academic activity (n =1,017) (n= 312) (n = 1,336) 
Arts/music 89.9% 88.1% 89.5% 
Computer/Internet skills 52.4% 61.5% 54.6% 
Coordinated school health services 17.0% 18.9% 17.4% 
Physical fitness/sports 92.1% 92.0% 92.1% 
Recreational activities 87.3% 87.5% 87.4% 
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APPENDIX B: 


STUDENT AND SCHOOL CHARACTERISTICS (CHAPTER IV) 


Table B1 


Profile of ASES/21st CCLC (Grades 2-8) Participants and Non-Participants (Student Characteristics) 


Number of Students 


Number of Schools 


Number of Districts 


Number of Counties 
% Female 


Race/Ethnicity (%): 
African American/Black 
Asian/Pacific Islander 
Hispanic/Latino 
White 
Other 


Eng. Lang. Class. (%): 
English Only 
I-FEP 
R-FEP 
English Learner 


Parent Education (%): 
College Degree 
Some College 
High School Graduate 
Less Than HS Grad 
Non-Response 

% Title I 

% NSLP 


% Student w/Disabilities 


% GATE 


Total in STAR 


Non-Part. Part. 
2,833,591 386,298 
8,313 5,034 
972 700 
58 54 
49% 49% 
7% 11% 
13% 8% 
47% 67% 
31% 13% 
2% 2% 
57% 40% 
8% 7% 
9% 11% 
26% 41% 
26% 12% 
19% 16% 
19% 23% 
16% 25% 
19% 24% 
47% 81% 
51% 79% 
11% 10% 
10% 8% 


Table B1 continued on next page. 
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Sample I 
Non-Part. Part. 
1,038,036 380,410 
3,053 3,053 
415 415 
54 54 
49% 49% 
8% 11% 
10% 8% 
67% 67% 
14% 13% 
1% 2% 
40% 40% 
8% 7% 
13% 11% 
40% 42% 
12% 12% 
16% 16% 
24% 23% 
25% 25% 
23% 24% 
77% 82% 
74% 79% 
10% 10% 
9% 8% 


Table B1 (Continued) 
Profile of ASES/21st CCLC (Grades 2-8) Participants and Non-Participants Continued (Student Characteristics) 


Total in STAR Sample I 
Non-Part. Part. Non-Part. Part. 

% New to School 40% 39% 40% 38% 
% Retained in Grade 1% 1% 1% 1% 
Grade Level (%): 

2nd Grade 14% 14% 15% 14% 

3rd Grade 14% 16% 14% 16% 

Ath Grade 14% 15% 14% 15% 

5th Grade 14% 14% 14% 14% 

6th Grade 14% 15% 15% 15% 

7th Grade 15% 13% 14% 13% 

8th Grade 15% 12% 15% 12% 
2007 CST ELA Results: 

% Prof. or Advanced A7% 31% 36% 31% 

Standardized Scale Score 0.06 -0.33 -0.22 -0.33 
2007 CST Math Results: 

% Prof. or Advanced 53% 40% 44% 40% 

Standardized Scale Score 0.06 -0.26 -0.16 -0.26 


Note. Sources: 2006-07 and 2007-08 STAR data files matched with 2007-08 after school attendance data. 
GATE = Gifted And Talented Education; HS = High School; I-FEP = Initial Fluent English Proficient; 

R-FEP = Redesignated Fluent English Proficient; STAR = Standardized testing and reporting. “New to School” 
means the student CDS code in 2008 does not match CDS code in 2007; “Retained in Grade” means the student 
grade-level in 2008 is the same (or lower) than the grade-level reported in 2007; “Standardized Scale Scores” 
are the CST scale scores converted to z-scores (M = 0, std. dev. = 1) based on the statewide mean and standard 
deviation for each CST subject area test. 
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Table B2 


Profile of ASES/21st CCLC (Grades 2-8) Participants by Study Sample 
(School Characteristics) 


Total in STAR Sample I 
Non-Part. Part. Non-Part. Part. 
Mean School Enrollment 686 742 765 745 
Mean 2007 Base API 765 705 711 704 
Mean School SCI 171 163 164 163 
School Location (%): 
Urban 43% 59% 53% 60% 
Suburban 54% 39% 47% 39% 
Rural 11% 6% 5% 6% 


Note. API = Academic performance index; SCI = School performance index. 
Cited from the CDE Public School Database and 2007 Base Academic 
Performance Index file. 
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